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ABSTRACT 
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compensate for deficiencies in currently available tests by shaping 
content, forrat and administration to the academic level and cultural 
background of the intended respondents. A battery of 13 test 
booklets, incorporating such characteristics as relatively low verbal 
levels, oral presentation and, in many cases, pictorial formats, 
dealt with areas of vocational orientation (pLars, aspirations, 
knowledge and interests); attitudes (toward selt, authority and 
social values) and job-related reasoning skills. Based upon analyses 
of item characteristics, reliability estimates, validity (against 
proficiency ratings), factor analysis of the battery and 
research- related findings, a number ot the measures were shown to 
possess a pattern of psychometric characteristics that warrants 
further research on their refinement and application, (Author) 
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Development ol' Evalfj.ation Measures for Use with 
XJeigh'borhood Youth Corps Enrollees 

Abstract 

A^'^ailable formal measures of the paper -and-pencil variety have generally 
been criticized as inappropriate for appraisal of individuals from so-called 
cultura.''ly deprived or disadvantaged backgrounds* This is esp?ecially so 
when a range of intellectual and attitudinal variables are to be considered. 

The present study represents an initial phase in the development and 
application of a battery of measures that would be suitable foi research 
and evaluation with disadvantaged adolescents in the setting of a youth-work 
training program* The paper-and-pencil instruments devised attempt to cc^m- 
pensate for deficiencies in currently available tests by shaping content, 
format and administration to the academic lev^'el and cultural background of 
the intcnd(-d respondents. 

A battery of IJ tesu booklets^ incorporating such characteristics as 
relatively low verbal levels, oral presentation ai^d in mary cas^s pictorial 
formats^ dealt w'ith areas of vocational orientation (plans, aspirations, 
knovdedge and interests); attitudes (tov,ard self, authority and social 
values) and ,iob-related reasoning skills. Based upon analyses of item 
characteristics, rei lability estiriates, validity (against proficiency rat- 
ings), factor analysis of the battery and research-related findings, a 
number of the measures were shovn:t to possess a patterr of psychometric 
characteristics that warrants fiu'thcr research on their iofinement and 
application. 
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Development of Evaluation Measures for Use with 
Neighborhood Youth Corps Enrollees 

I o INTRODUCTION 

A. Purpose of the Study 

The importance of systematic evaluation in gauging the effectiveness 
of any broad program of social change seems axiomatic. However ^ for such 
programs^ usable quantitative tools to assess and describe those changes 
that reflect desired goals have often been found lacking. Or^ more likely^ 
available measuj'es are considered inadequate as a basis for judging many 
areas of program performance. This appears to be equally so for the class 
of social prograias that fall under the rubric of *Voi 3 .th~work training'^ and 
especially for those categories of goals concerned with changes in the 
abilities^ attitudes^ or adjustment of the individuals served. 

Sweeping criticism of youth-work training programs has been leveled 
by Herman and Sadofsky (1966) for a general failure to provide the scrutiny 
and potential modification of program components that derive from continual 
efforts in evaluation and research. Dubner (I965) is more specific in out- 
lining the need for tests and measures of individuals in the many phases of 
a training program for adolescents from a paverty level culture. Only a 
superficial reading of the sections on evaluation, reported for hundreds of 
employment -training and work-study programs in the congressional report on 
^’Profile of Youth - I963" (U. S. Senate Committee on Labor and Public 
Welfare j 196^^); provides di*amatic evidence of a reliance on ^^testimonial" 
evaluation,' There is a distinct avoidance of quantitative or systematic 
assessment of either the total program outcomes or of the effects of these 
training programs on the youth involved in them. 
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rhere may be some question of whether such criticism is fair and 

r.hr expectations for definitive research and evaluation with youth<' 
work programs are entirely realistic. Can good intentions be fulfilled if 
th-ve are fcrirddable limitations in the availability of measures which v;o'old 
be suitable foi- conducting the desired assessments? In the area of iridivid- 
uax pf rformarice rTieasurement^ the question is highly rel evant - -pa'j'ticularl.y 
when one considers the population of adolescents for whom youth -w'ork programs 
art intended and the characteristics of those formal assessment (‘'psyehologi- 
*6,1.’; instrujn^nts that are currently available. 

fr->b.,ems arise, largely because of the specific conter.t_, format v, ^ 
standardisation oT these measures when applied to individuals whose sonnwhat 
unique cuxturai experiences, motivational patterns and skills are shaped by 
a pow.rty- le'/el background and who^ in many cases, are also minor'ity group 
m-.mb' ro. Va“;icus inadequacies in available tests ^ as utili 2 od with member s 
of iriincrity cr disadvantaged groups^ have generated considt-u-abl ;r p>rofe ssacnaJ 
corice:*Ti (Campteil^ 19^1 ; New York City Commission on Hurrian fights, I968; 
S.?S31, l>;;f6'0- strong implication of this concei'n is not only sk^ oticism 
about existing test materials ^ but a possible need for diffei^nnt measures 
approprlatrly tillered in content ^ administration and iuterpr- tation fer use 
with such groups. In considering the evaluation needs of a youth -vrork train- 
ing program- with its many specific iioals that involve charges in individual 
perf ormance , any implication of a need for new approaches tc measure. ment 
takes on added significance. 

the largest of the governmental! y supported youth- work training 
progrdums , the Neighborhood Youth Corps (N.Y.C.) must obviously be concerned 
with thv availability send application of individual assessment t? :hi',iqj. s in 
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the conduct of research and evaluation efforts. The techniques must be 
" relevant and reasonably a-ccurate since the means by which program accom- 
plishments can be demonstrated are certainly no better than the quality 
of the tools used to assess those accomplishments. The present study^ in 
general^ focuses upon problems of measurement and evaluation associated 
with adolescent school dropouts whose status is customarily defined as 
"'disadvantaged'' or "culturally deprived" and for whom some form of work 
experience and vocational counseling are being provided in the setting of 
a youth-work training program. More specifically^ this report is intended 
as an initial or preliminary phase in the development of a battery of 
measures which would be particularly suitable for evaluation of male and 
female Neighborhood Youth Corps enrollees. These measures are to deal, 
with characteristics of enrollee performance that reflect a variety of 
youth -work training program goals and mighty therefore^ be expected to 
change as a result of the individual's participation in an N.Y.C. project. 
Results of initial field-testing of the battery and examination of the 
appropriate psychometric characteristics of each measure are to be con- 
sidered in recommending the ones most suitable for further development and 
use with the N.Y.C. program and^ hopefully^ for use with disadvantaged 
adolescent groups in general. For this study phase^ it should be understood 
that only readily available and limited^ concurrent criteria will be uti- 
lized as well as consistency estimates based upon single test administration. 

B. The Neighborhood Youth Corps Program 

(l) General Description : In order to understand the choice of vari- 

ables and the design of the measur-es used in the present study^ it is 
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necessary to clarify the overall scope and purposes of the Neighborhood Youth 
Corps program in carrying out its youth-work training function- 

Very briefly^ N-Y-C- operates a nationwide federal program administered 
by the U- S- Department of Labor and the Office of Economic Opportunity for 
the purpose of providing job experience along with vocational guidance or 
counselirig for male and female adolescents^ 1^ to 21 years of age’^‘‘^ who come 
from families with poverty- level incomes. These adolescents may either be 
ful.l-time st’odents in an "In-School" program or high school dropouts in an 
'^3ut“*of “--School’' program. Sponsors for N-Y-C. projects are prima’riiy local 
eomm’unity agencies or othei? nonprofit organizations which provide guidance 
Services, develop jobs in local coimmonity organizations as well as their own 
organization — -and assume overall responsibility for administrative services 
in the conduct of their project. (Fqr a detailed summary of the program and 
its operation see U- S* Department of Labor;, I 967 - ) 

Of major relevance for any evaluation effort are a number of character- 
istics that would be expected in a program which is national in its scope 
and geographically widespread. These include not only substantial variabil- 
ity in the types of agencies that serve as sponsors^ but also in the training 
and background of professional personnel who work with the enronl.ees; in the 
type and level of jobs available for the enrollee and in the efiectiveness of 
on-the-job experience provided. For example^ enrollee jobs may range q.uite 
widely from tasks that involve conservation work in a local park to those of 



The age group which the program served at the initiation of this study 
was the 1.6-21 age group. This was changed to include the yo'onger age levels 
in July of 1967 and although the measures used in this st'udy were designed 
with the original group in mind^ the 'wider age range is not considered as 
critic8-l to the potential application of these measures. 
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an assistant in the electronics repair shop of a military base* Although 
each enrollee is paid at least the federal minimum wage by the N.Y.C. project^ 
there may be variation in the number of weekly hours of work permitted and 
thus^ in the weekly wage paid at any given project. 

Variability in the composition of the population served also contributes 
significantly to any evaluation considerations* N-Y. C. enrollees include a 
variety of ethnic and racial groups from rural and urban areas who^ for 
example^ may be Caucasian^ Negro^ Puerto Rican^ Mexican or American Indian* 

The differences^ stemming from varied language and cultural backgrounds^ are 
apparent in their effect upon an evaluation effort. More subtle variations 
in the enrollee population from project to project may be introduced over 
time by social changes that are not the direct result of project administra- 

f 

tion or overall operation of the N.Y.C. program* One such change that can 
have a direct effect upon the characteristics of a large proportion of males 
available for the Out-of -School program is that of military requirements and 
criteria for induction as defined by the Selective Service System* Other 
social changes involve the initiation and expansion of various governmental 
or private youth- training programs which can operate to draw off selective 
segments of the disadvantaged adolescent group and change the composition of 
the population available to the N.Y.C. projects* Changes in the local job 
market can have an obvious effect upon N*Y*C. enrollment and there are even 
variations from city to city in the acceptance of government anti-poverty 
efforts among minority group members. Such community attitudes can and 
probably do affect the recruitment and retention of enrollees* It is of 
importance that such program variations at least be understood and that any 
evaluation variables chosen cut across as many subclassifications^ or 
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prcg..ajn differences, as feasible in order to design a workable evaluation 
system- -as well as to understand its limitations. 

(2) Pertinent Enxoll:^ Characteristics ; Since the present study is 
concerned with the evaluation of adolescents enrolD.ed in N.Y.C., some addi- 
tional aspects of the relation of the enrollee to the program and his indi- 
vidual characteristics are important in shaping what is to be measurvd for 
evaluative pui’poses, and how this is to be accomplished. 

An initial consideration is that the adolescents^ with whom the results 
of this study deal, are school dropouts. Although the N-Y.C^ In-School pro- 
gram is intended to serve those who are full-time students and to assist 
them in I'emainl.ng in school, the disadvantaged adolescent group for which 
available tests and measures are least likely to be applicable is the ore 
with the greatest ’'handicaps*'--!. e . , the unemployed dropout, dlius, the 
probJems of evalubtion, as will be shown in subsequ. nt discussiorij are most 
pressing for this group. A chai's.cteristic of the dropouts that is important 
in shaping evaluation inst rumen Ls is ’•'heir re.lati’/ely lo^w verbal Ic'/el.-- 
particulariy for males. Average reading level is found to be slightly above 
fifth grade for the irule enroll^?es \-/ith the low point of the i ange at litt]-: 
more than second grade, Ihe mean value for fcKialts is apprc/imairr v a grade 
level Higher.* Ihe largest proportion of the enroll a>’c ycu.ng adolescents^ 

These conclusions shou2 ' be considered with caution since they are 
based upon limited evidence c ns is ting of: (l) a sairiple of 100 male and 100 
female scores on the Icrwa Achievement Test administered at an Urban Youth 
Of portijLP.ity Center and (2) conversations with professional personnel at IJ Y-C. 
projects and at a Youth Opportunity Centtr seivi/ig a major ^*-ity. No adequate 
national sarriple of reading scores is kricr^Ti to be a.ailable for this pdr'-iculcii 
adolescent population. 
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primarily in the I6-I8 year age range ^ with males having left school at about 
the 10th grade--often for academic and/or disciplinary reasons and an 
expressed ”lack of interest*' in school. Most have never been err^loyed in 
full-time positions before. Females have generally left school at about one 
grade level higher than males and less often for academic and disciplinary 
reasons than for financial reasons and requirements for child care. Their 
lack of job experience is found to be even greater than for the males 
(Tucker^ I966; Walther & Magnusson^ I967). 

Enrollment in N.Y.C. is entirely voluntary on the part of the individual;, 
as is the decision to remain. Enrollees may leave at anytime they choose^ 
whether because of dissatisfaction with the program or simply because suit- 
able full-time employment is found. No specific period for remaining in the 
program is mandated so that the time spent by an enrollee in an N.Y.C. proj- 
ect is highly variable. However^ about six months is considered (very 
looseiy) to be a "typical.*' maximum^ although this limit may not be enforced 
by any given project.. Thusj there are no firm criteria as a basis for defin- 
ing completion — whether in terms of specific ski. 11s developed ^ employment 
obtained^ or time spent in the prog:ram. 

There are typically two adults with whom enrollees have their primary 
day-to-day contact and who assume major responsibility for the enrollee*s 
progress during his stay in the program. One is considered a guidance 
counselor at the N.Y.C; project facility and the other is a work supervisor 
at a job site. The former individual is an employee of the N.Y.C. project 
and the latter is usually an employee of the agency which has agreed to 
accept the enrollee on a job. Counselors often maintain close liaison with 
work supervisors in order to deal with any common areas of concern regarding 
the enrollee and for purposes of evaluating his progress. 

>■ ■ 12 
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C. Program Goals and Ob;lecti‘ves 

Clearly^ an evaluation system must reflect in some meaningful -way the 
overall or long-range goals of the program under evaluation and, perhaps^ a 
niomber of more immediate objectives theit logically lead to those desired 
goals. In the case of N-Y-C.., as for other youth -work training programs, 
the goals have tended to cluster ‘under an "umbrella” criterion referred to 
as "employability” (U- S. Office of Economic Opportunity^ 1967)- Any out*-* 
comes of youth -work training efforts defined in terms that deal with obtain- 
ing and holding employment are certainly defensible- But if such outcomes 
are too narrowly defined and linked only to specific measui^es of eni'ollee 
employment following participation in LT.Y-C. (e.g.^ time required to obtain 
employment,, job tenui*e^ salai^y^ etc. ); then the evaluation system I'isks 
placing too many eggs in one "criterion basket." Such overall perfoimiance 
criteria can fail to account for important changes in the in d :ivldr -^L 
enrollee -which are brought about by TT.Y.C- pac;ticipation ^dd linked to 
ultimate employability^ and thus can also fail to identiXy those specific 
components of the program which need stre'-igthening or modification. More 
explicitly^ an E-Y-C- project may be carrying out its intended functions 
quite adequately. However^ in its pa:)i*ticular locale,, the job market may be 
chronically poor^ biases more e’vldent against employment of minority group 
members^ or the level of available jobs exceptionally law. If post-N.Y.C» 
employment criteria are the sole measures upon which evaluation is based;, 
then obviously benefits reflecting indiiridual change as derived by the^ 

K 

enrollee from the DI.Y.C. project^ are almost certain to be overlooked- In 
essence^ there are advantages in assessing a variety of changes in individ- 
uals^ as well as those broader program goals that are customarily used to 
define a criterion of employability. 
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Specific short-term progi^am objectives that are reflected in changes in 
enrollee be.ha\loi‘ constitute the basis for selection of the measurement var- 
iables considered in this study. Conversations with N.Y.C. program personnel 
(e.g.;, coionselors , project directors)^ N*Y-C. headquarters personnel at U. S. 
Department of Labor and examination of the limited research on training 
programs for disad/antaged adolescents lead to reasonably good agreement in 
defining many of the objectives that are seen as important. Five such areas 
(which arc recognized as having considerable overlap)) can be defined grossly 
in order to provide a convenient framev;ork for the content and development of 
specific measures. 

These are summarized as follovTs: 

(1) Vocational Orientation : This category of outcomes is concerned 

with the individual's general appreciation of the v;orld of v:ork. fob exper- 
ience and vocational counseling are expected to p^'cvide him with a better 
awareness of what kinds of jobs night be available^ the backgro'und , experi- 
ence and training that these >'equire and uhat a job involves in terms of its 
duties or demands on the woj*ke7u As part of this, there seems to be an 
attempt to develop realistic perceptionc of jobs that v:o.nld be reasonably 
attainable and to have the enrollee shape his fixture plans accordingly. 

( 2 ) Skills in Seeking and M ai ntaining P' mpl o yment : A considerable 

amount of effoi’t is apparently exj^ended by counselor.-' and work supe :viso7's 
in instructing the enrollee on haw to go about finding a job, and hew to 
behave once he is employed. Such instruction may include where to look for 
jobs, hew to make ap>pl ication, how to conduct oiusclf duidng an into i view, 
proper dress or grooming on a job, and ha; to bcha*v*e tOvvard. a supcin Lsoi 
and one's fellow employees- -in essence, the lu.les of conduct g*:nc rally 
accepted on a job; as amc rphous as these may bt in many instances. 
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(3) Attitudlnal Factors : There is significar:t stress in youth-work 

prograjns placed upon developing ”propex' attitudes" on the part of the e:: 2 rcllee- 
These range from attitudes tofw^ard oneself to attitudes toward society in gen- 
eral^ and especiall;^^ tofward authority figures enccuntered in the society. The 
rationale is "based upon the obvious ass*jmiption that e;nhoi*::.ed self-esteem is 
valuable for developing an individual’s aspirations and for *jltimte adjust- 
ment in the vocational as -weU. as persoriail.-social areas that, in addition^ a 
certain degree of control of overt hostility towai'd authority is essential for 
maintaining employment;, as •well as for keeping out cf trouble (primao*ily 
legal) in various other spheres of personal and social actlTity. A group of 
closely related attitudes that many counselors also seek to instill ac*e a 
sorriewhat broad pattern cf behaviors based upon a mllingnevs to plan ahead. 
These axe best typified by such activities as handling fina,nc-iS (-.-g.^ saving 

i 

money); making sacrifices for future educatlcn and recogniu^ing the conse- 
quences of hasty aggression toward others and the legal, difficulties that o:ari 
follow. The pattern is one that sociologists have customarily defined as 
"deferred gratification" (Schneider & Lysgaai'd. 1953; Straus. I 962 ). 

(^) Job Motivation and Interest Patte:rns : j^lnothe::’ impcrtac'it but some- 

what different facet of attitude development is geared, t-rward shaping motiva- 
tion of the e:xrollee ixi a way that leads him to see a possible "paycf.t'*‘ in 
working arid in striving for job advancement. This may be refle:ited in a 
desire to go on for additional training that leads to better jobs, a will- 
ingness to put more effort into the job itself, in seeking employment or in 
conforming to rules on the job. 

Since patterns of interest in job tasks are also bei.ieved related to job 
motivation.; one of the intentions of a program tha.t offers work-training 
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experience and vocational guidance is to enhance motivation by helping the 
individual to define and clarify those interests « Ihius^ there is an assump- 
tion that^ through experience and vocational information;, the enrollee 
might crystallize or sharpen his vocational int e-rests --perhaps even to 
"upgrade^’ the level of job tasks that form his interest pattern « 

(5) Practical Reasoning in a Job Sit ua tion : This final category of 
objectives is probably stressed most frequently in a vrork- training situation 
by a job supervisor who expects the trainee to "use his head" or ‘‘stop end 
think" when carrying out some assigned task. At leasts the intent is that 
he pay carePai enough attention to directions and use reasonable judgment 
in carrying out work assignments* Tfils fom of practical judgment on the 
job undoubtedly encor^passes many attitudinal and motivational components 
rather than being purely an aspect of what is usual.! y considered 'basic 
intelligence." Personnel in work-training programs certainlp^ behave as if 
the characteristics that define these qualities can be instilled through 
training. Thus^ it appeal's v;orih considering a set of job -related; reason- 
ing skills which may possess unique dimcnsiofis that are subject to change 
as a result of job experience and jroper guidance. 

It should be apparent that the fairly cursory discussion of po&sibiv 
H.Y.C- program goals outlined here in no way ey.hausts a rnomber of other 
possibilities; or denies their importance in youth-v;ork training programs. 
Remedial training in aritlsnetic and verbal skills is ^undoubtedly an ar t a of 
value as are any number of teclv ical (vocational) skills that may bt imp'or- 
tant to job performance. It is simply a fact that neither remedial shills 
instruction nor the formal dc\^clop>mont of specific technical abilities 

. lej 
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appear as uriiform requirements for even a majority of the N.Y.C. projects.* 
In any events the sort of objectives outlined above are believed — by a 
number of professionals in the youth -work training field- -to be at least as 
important tc later job success, for economically and o’lltxirally deprived 
adolescents; as are sp>ecific remedial and technical skills. It is also an 
apparent ass'-uription that these sorts of objectives are not only modifiable 
as a result of participation in N.Y.C. , but important aspects of enroll'^e 
performance in that they can influence (i.e.; ai'e correlated with) post- 
program job adjustment or success. ITse remaining sections of this repoi-t 
are intended to translate these broad objectives into measurable terms and 
to demonstrate the uti.'Lity of measurement devices that reflect such o'^jcic- 
tives . 

D . Measure m ent Pro b J.eras with Culturally Deprived Groups 

Ceruain problem areas associated with measurement as it relates to dis- 
advantaged adolescents (often dealt with in the context of minority groip 
merribership) nro.st be taken into account, before suitable evaluaticn measures 
for this population can be chosen, dcsignr.d, or utilized. Failm'e to dc so 
is, in fact, the basis of much of the criticism aimed at c’urrently available 
formal tests when applied tc cultui'ally deprived groups of any age level. 

ITie most recent intciu'st in these problems of testing has derived from the 
industrial st.tting and involves questions regarding the applicability of 
available tests for employment selection pirposes when applied to minority 
groups and/or cuit\uaily deprived applicants. Discussiojis of possible 



ITiis was unquestionably the ease at the time this study was initiated 
and stiJJ. remains Jjtngely tr ue - -alth >ugh there has been an increase in the 
number cf r^miedial skills programis undertaken by N.Y.C. projects. 
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disc riminat or y effects of testing in personnel selection have been presented 
by Guion (I966, 3.967) and by Kirkpatrick^ Evren^ Barnett and Katzell ( 1967 )* 

The problems as viewed by industrial psychologists center aiound dis- 
criminatory bias of presently used measures^ on the basis of their differen- 
tial test validity (i*e.^ some tests may be valid for whine individuals^ but 
have little or no validity for Negroes). Some indication that this is a 
legitimate concern has been demonstrated by Kirkpatrick et al. (I967) and 
Lopez ( 1966 )« ITie types of solutions proposed for dealing with this form of 
test bias- ''given the existing measures --have involved separate analyses and 
interpretation of test results for different ethnic or cultural groups (e.g. 
use of different regression lines). 

yVich earlier recognition of unique evaluation and testing problems for 
individuals from economically and culturally deprived backgrounds has corae 
from the field of educational evaluation. Formal techniques in educational 
appraisal of the abilities of disadvantaged children have focused almost 
exclusively upon intellectual or cognitive functioning ^ with continuing 
attempts made to overcome deficiencies in existing measures by resort to 
"culture fair" materials (Anastasi^ 196b). These attempts to achieve cultur 
ally "neutral" cc>nte:it in measures of intellectual skill have been sevc-rtly 
criticized and considerable skepticism rerriains regarding their utility in an 
educational setting. V/esman^s (1968) discussion of some "follies committed" 
in attempts to de^velop cultui-e-f ree instriLments is especially relevant. 

The point to be stressed^ hoviwer^ is not only that the pattern of 
educational te‘ ...ng for culturally deprived youngsters has been concerned 
almost exclusive.’.y with a search for intellectual indices, but as is the 
case for the industrial setting much of the research in this problem area 
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has been dependent upon scores on existing intellectual or achievement 
measiares to demonstrate differences betveen ethnic groups or to correlate 
test performance with degree of cultural deprivation {Coleman^ 1966 ^ 

Deutsch & Brown j 196^+; Dreger Sc Miller^ i960). Paradoxic ally ^ the tests 
used are often qm^stionable for application to disadvantaged, groups in the 
first place. The finding of such differences is not particularly surprising 
(although to some extent worth demonstrating in order to confirm their 
existence). A more logical approach would appear to reside in recent at- 
tempts to determine why the differences exist in order to facilitate the 
education of culturally deprived youngsters, and to understand the test 
problems. There are findings, for example, of differences in background 
characteristics of the disadvantaged that result in different learning 
"styles,” which in turn can affect test perfor:nance on a variety of differ- 
ent measures. Karp and Sigel (I965) in their review of research on testing 
disadvantaged children highlight what they term such "new approaches in 
appraisal" and cite suggestions by others (Gordon, I965; Lennon, 196^) 
which carry strong implications that some new directions in test development 
are needed- -not merely variations in interpretation of currently available 
measures. This implication is certainly apparent in the conclusion reached 
by Karp and Sigel (I965) that; "Psycho-educational appraisal of the disad- 
vantaged pupil confronts us anew with the need to develop assessment proce- 
dures that both clarify the mechanisms by which learning occurs and guide 
the teaching-learning process." 

Without unnecessarily belaboring an extensive literature, that has only 
tangential value for the immediate purposes of this study, the argument to 
be pursued here does not question the fact that the differences betveen 
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culturally deprived and nondeprived groups do exist for a mde variety of 
formal psychological instruments and especially in certain cognitive- 
intellectual areas. Bather, it is that these measures are generally 
developed for middle-class individuals with their content, format, tech- ^ 
niques of administration and normative data intended almost exclusively 
for such a group. Thus, results of testing with such measures merely con- 
firm, in a broad sense, the very pattern of those differences that are a 
logical outcome of the cultural deprivation imdergone by the disadvantaged. 
Recognition (or acceptance) of this assertion does not^ in itself provide a 
solution to the problem of usable measures for appraisal purposes--whether 
in meeting the evaluation needs of a youth -work training program or in an 
employment selection situation. Nor does reinterpretation of the results 
of existing measures for different groups necessarily circumvent or solve 
the problem. Any application of commonly used measures oyer the wide cul- 
tural and verbal range of the poverty level subpopulation ^ that is customar- 
ily defined as "disadvantaged,” seems open to question. One wonders, for 
example, whether any customary interpretation of test results can be 
sensible when a formal paper and pencil, verbal measure is applied in stand- 
ardized fashion to an adolescent or adult whose reading level is near func- 
tional illiteracy and who completely misinterprets the content, or the 
specific intent, of a large number of the items. 

The essential criticism of available measures and the rationale for 
considering new approaches might be stated as follows: any test obviously 

represents a sample of individual behavior and this behavior sample can be 
considered adequate only if the stimulus materials (i.e., test items) which 
elicit that response serve to communicate their meaning in reasonably clear 




fashion. Where such communication is essentially ineffective ^ the value of 
the response elicited is likely to he of limited applicability for any 
assessment purposes (e.g.^ no one argues against dropping "ambiguous*' test 
items). In this lights a major conclusion that one can reach with regard 
to testing the culturally deprived is that there has been a failure to take 
account of their verbal skills oi unique background and cultui-al character- 
istics in shaping test format ^ content and adiiiiuistration--so as to devise 
measures which,, at the very least,, comrmjinicate in the manner intended by 
those who itilize and interpret such measures (i.e.j a 100 item attitude 
measure^ entirely verbal in format ^ administered to an individual who cannot 
read a significant proportion of words and who lacks familiarity with much 
of the descriptive content may serve to verify the limitations of his par- 
ticular cultural background^ or ve^rbal inadequacies- -it is unlikely to be a 
measure of the attirude dimension (s) intended). 

True^ this is not the only problem found in testing minority and 
culturally deprived groups. The contention here is that it is a major one. 
Other specific aspects of background must also be consider:! in relation zo 
tests and test-taking behavior before a suitable set of ground miles can be 
formulated for choosing or designing measures that would be applicable to a 
disadvanteiged adolescent group. Continued trial and error with available 
measures j to see which of a vast number may be applicable for a wide variety 
of assessment needs ^ is not only laborious but may be largely fruitless* In 
light of even tlie limited knowledge of background characteristics^ prior 
test performance and special problem areas in testing culturally deprived 
adolescents, one should be able to narrow the field and defiiie relevant 
measiu'ement techniques that have a reasonab.le likelihood of proving successlb 
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Those considerations^ and their re3.ation to tests and the measurement process 
which should be taken into account include: 

(a) A relatively low verbal reading level (as has been pointed out 
previously). This constitutes a major characteristic which limits the com- 
munication process and choice of test materials. It does not mean that the 
group is "non-verbal” in any sense. Even a third or fourth grade reading 
level provides a range of usable verbal skills --reading and oral — and does 
not necessarily imply that ideas must be communicated in an exclusively 
pictorial mode. However^ existing measur'es designed for use with middle- 
class adolescents j that might also deal with suitable evaluation areas for 

a youth-work training program^ are likely to possess too high a verbal levels 
whereas^ measiJires having lower verbal level material are likely to be too 
childish in content for an adolescent. 

(b) Performance of disadvantaged groups on intellectual ability mea- 
sures is found to be superior in certain respects when oral presentation is 
used (Graham 8c Orr^ I966). ThuS; comprehension for groups with low reading 
levels may be enhanced by utilizing oral communication along with any verbal 
printed materials. 

(c) Differences in language meaning,, or connotation,, may exist for 
common words and phrases among minority group adolescents. Therefore^ there 
is potential value in being aware of unique slang expressions and idiomatic 
modes of speech that can be highly meaningful to an adolescent or minority 
group culture — but not generally found in widespread use among middle-class 
adults. Similarly, in the area of communication and test content, there is 
the consideration of specific cultural experiences that are commonly found 
among middle -class youngsters but are relatively unfamiliar to a lower-class^ 
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rrdnority group child. An obvious example involves choices of hobbies or 
avocational activities used as items in interest measures. These may be 
clearly understood by middle-class adolescents but because of economic con- 
sideration axe often out of the realm of prior experience for the disad- 
vantaged. Even mth use of pictorial material in tests intended to minimize 
such criticism the content is often found to represent middle -class activi- 
ties and to show individuals with strictly Caucasian racial characteristics 
(Ceist, 1964; Vleingarten^ I 956 ). 

(d) For the school dropout from a poverty background there is a complex 
pattern of negative attitudes t^ooard academic trappings^ of which formal 
testirig is one. The general history of failure — and the individual's con- 

inuing awareness of it- -is coupled with inadequacies in prior test perfor- 
mance on measuies which have repeatf'-dly exposed Viis poor verbal skills. 

There is little relation that such an individual can see betw'een testing and 
v/hat it means him, It has ^ for the most part^ sei*ved merely as an annoy - 
ance with no discernible ‘‘payoff'.'^ In fact, it my even have ser/ed as one 
of a nuirber of barriers encountered in seeking e]:iployment. 

(e) Since test-taking does not tend to be viewed fa/crably by disad- 
vantaged adolescents^ competition in such an area of pej’ formiar.ee is rrdlninized 
so that the indi^adual cl jcs not alv/ayp, expend the effort to ”do his best.’" 
I\u'th?rmore , much of the development of test-taking skills ( "test-wiseiiess " ) 
stir.ulated in schools with ])redo:ninantly middle -class white jaipils (and by 
parental xrcss'orcs) is mi limal for cultuially deprived and minority group 
pupi3r. n-ic logical result of such experience is a response to -^^sting that 
is often subsumed luidei' ’’poor tt- ,.t-taking motivationh' lengthy tests with 
numt;’'ou5 verbal items that are not only poorly uriderstood^ but ■inint..- resting 
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in content^ provide little to command attention and only contribute further 
to low motivation levels. 

More subtle and compi.ex problems of motivation can arise ^ which may 
influence test response by minority group members. Peelings about racial- 
and/or class distinctions between examiner and examinee and partic'clarly in 
the realm of conformity and response to an authority figizre have been sug- 
gested as influences on test scores (Dreger & Miller ^ i960)- • 

(f) An effect upon test results ^ when law SES or disadvantaged groups 
are compared with middle-class youths is the relatively lesser degree of 
consistency of response among the former group (SPSSI^ 1964). Although this 
low reliability has been attributed to daily environmental factors that stem 
from economic deprivation, and lead to response instability^ it is s^jggested 
that some of the variance may be accounted for by characteristics of the 
tests utilized and by content that stresses middle-class cult’or^al experiences. 
The problem for consideration is one of whether items chosen as culturally 
relevant would reflect a greater range of meaningful, variability than avail.- 
able formal measures have tended to uncover. 

E - Available Formal Measures 

Based upon an extensive number of measures that have been pub 11 she d^ and 
others devised by researchers for utilization in jisychological and sociologi- 
cal. studies^ it is legitimiate to question whether any of these may be directly 
applicable to evaluation of disadvantaged adolescents in a youth-work training 
program- The number of available tests is immediately reduced to more manage- 
able proportions if two critical requirements are imposed; first;, that they 
be applicable specifically to the N.Y.C. program objectives and second that 
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they be suitable for a population of culturally-deprived school dropouts. 

If in addition a third necessary requirement is imposed- -that the instru- 
ments possess at least a reasonable number of the qualities customarily . 
attributed to formal (usually published) measures --then the field narrows 
to a point of uncoimfortably limited choice. 

By formal measures (as contrasted with informal^' measures or question- 
naires) it is generally meant that certain qualities exist in standardization 
of directions^ adimlnistration and scoring; that there is information on 
statistical properties of items and total scores; that these psychometric 
characteristics show a pattern which meets acceptable "professional'' cri- 
teria and, finally, that the measure is applied widely enough to provide 
normative data for the population of interest and, whei’e necessaiy, for 
appropriate subgroups within that population. (For a general s-ajiLmary of 
formal ai:id inforrrial ineas\irement see Greene, 195^, Ch. 2.) 

Those available formal measures that might be considered to have pos- 
sible applications in a group setting, for N.Y.C. evaluation purposes, often 
fail to be sufficiently compatible v;ith the characteidstics of the cultur- 
ally deprived as discussed above. Among the reasons are, I'ii’st, because 
they are found to be entirely verbal in fonnat and at relatively high 
reading levels (in terms of items and directions). ITiose Aw that ai'c 
pictorial measures are exclusively so and fail (for j’casons unexplained) to 
enhance their infortration by combining verbal and pictorial material. In 
addition, the content (pictorial or verbal) is often culturally biased, 
dealing primarily with middle -class concepts and experience. VJhcre lov: 
verbal-level material is used, the content reflects the experiences or 
interests of young children. Hccogiiition of this j Lmitatic^n in iu;ely 
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intellectual measures has led one publisher to deal with the content of 
arithmetic and verbal skills items in a context that depends upon more 
comivon adult activities or experience - -ev^en tho^j^h the material is at a 
low verbal reading level (Karlsen^ Madden 3 : Gardner, 1967)^ 

Another mjor shortcoming is that the formal measures are often rela- 
tively lengthy (90 to 100 items or more for a measure), t/hen combined with 
an entirely verbal format and poorly comprehended, or experientially irrele- 
vant, content large numbers of items covering the same content area only 
add to the easy distractability and poor test -taking motivation attributed 
to culturally deprived school dropouts. 

Additional contiibutions to the limitations of many formal measures 
are found in featmes that include tight time restrictions (especially 
vfhen applied with noncompetitive individuals v^ho are not ”test-v;ise'' ), 
Tiumerous items arranged un a sheet where sequence may be confused or where 
items are easily skipped over and, finally, errors engendered by even the 
customai'y process of tr*ansposing a response from the item choices on a 
page to a sej'arate answer sheet (Clark, I9CS). ihes'' and similar require- 
ments, normally desirable for ease in the atoinistration and scoring of 
fori.'.al group reasu.res , lead to a remoteness of the examiner from the individ- 
ual r‘'spondcnt in the test-taking process. For disadvantaged adolescents 
vrho v;ill often fail to ask foi’ clarification or assistance where appu'Opriat': . 
such highly structured al’ninistrat ion may add \aine cessarily to eriors of 
measurement. 

Tn the area of i n f o i 1 m>-- a s u r erv~ n t , vrhich derives largely from itScoi 'h 
with disadvantaged groups (moim often in tei’ins of a low SF:s categoiy). tie. 
ins ti aments used are usually drav:n up' by a rLoearcht r for his rarticulaT 
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study. A considerable number of such informal measures (especially noncog- 
nitive ones) that have been used with disadvantaged groups are readily found 
in the literature. Although on the surface many such paper and pencil mea- 
sures used for research studies appear applicable^ their major drawback is^ 
inve.riably_, that by no stretch of the concept can they be considered 
"measures." 'fhey are more accurately termed "questionnaires/’ or a compila- 
tion of questions which are assigned scale values ^ hopefully with some ordi- 
nal properties j but with little if any analysis performed to indicate their 
measm'ement characteristics. Study results and conclusions are then reported 
as if measures of known characteristics and proven capability have been 
utilized. Any of the inconsistency that already exists in research results 
concerning disadvantaged adolescents can only be magnified by a failure to 
develop measures with known properties that are consistently measuring 
reasonably understood dimensions of behavior. 

Although there were shortcomings found in both available formal ana 
informal measures which would preclude their direct application as evalua- 
tion indices for disadvantaged youths, a number of these nevertheless provided 
important clues to content and item format 'hat were incorporated in a n>juvibcr 
of the measures developed for this study. A survey of many of these applica- 
ble test , materials is reported in the draft of a specifications document pre- 
pared in the course of developing the measures (Educational Testing Service,, 
1967). Some portion of this backgrouTid material is sumnarized in the 
description of the measures that appears in Section II of this report. 

K . Evaluat i on vs. Fi^ediction 

In developing any form of behavioral measure , an appreciation of its 
intended use(s) bears emphasizing. There can be important implications for 
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data analyses and the decisions reached in test construction^ when tests are 
considered for single or multipurpose application. The issues involved in 
test utilization as a function of measurement properties are fairly complex 
and arguable. In order to confine this brief discussion to some of the more 
pertinent issues for the present study^ the assumption made is that the pri- 
mary purpose of the measures to be developed here is one of evaluating cer- 
tain N.Y.C. enrollee characteristics over the period of months that the 
individual remains in the program. Given this assumption^ it follows that 
if the numerous program objectives for N.Y.C. ^ as outlined above^ could be 
translated into highly precise forms of enrollee behavior- -that are readily 
observed or measxored — then the items comprising a given test could be 
formulated as direct statements of such behavior or performance. Any mea- 
sure containing items that define such specific aspects of performance would 
be considered valid on the face of its content ('*face validity") and it 
might be argued that there should be little concern with item characteris- 
tics^ or test reliability and predictive validity. Face validity invoked 
for evaluation purposes merely says that the enrollee *s test reponsss^, after 
some defined degree of participation in N.Y.C- ^ are expected to reach given 
levels and the closer he comes to reaching these defined standards the 
greater the success of the program. Thus^ the evaluation of change repre- 
sents ^ in effect^ shifts toward or away from the desired behavior. This 
sort of argument for "representativeness of content" as the primary basis 
for design of an evaluation measure has been advanced by Horn (I968) and 
serves to minimize reliability (or predictive validity) as a consideration. 

In practice^ unfortunately one is rarely lucky enough (or gifted 
enough) to be able to define test content and desired individual responses 
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so clearly- -least of all in a social program that has rather broad goals 
and subgoals invol-vlng individual impro-ij-ement . There is simply not enough 
known about the specific individual behaviors to be changed (or the form 
of test items that -would directly reflect such behavi.ors)- And^ more 
important^ there may be nagging suspicions that what are thought to be 
desirable outcomes in enrollee behavior at the completion of the K»Y.C- 
program may be relatively unrelated to longer range (i.e._, post progi‘a2n) 
goals. Where this possibility exists as it usually does — and the N.Y»C- 
program -would appear to be no exception — there must be some attention paid 
to reliability of the measures (or its estimate based upon internal con- 
sistency) as well as some aspects of predictive validity, jllso as Ebel 
(1968) has pointed out in defense of such a position^ tests intended for 
purely evaluative purposes often end up being used for decisions that 
affect individuals j for example ^ by an N.Y.C. counselor who utilizes 
them for guidance pur*poses or for assignment of eni'oHees to gob sites. 

-^^The point is that it can be reasonable to have a multi])unpose test 
battery in which measures that serve an evaluative function may also serve ^ 
to some extent^ a success:ful predictive or classification puirpose. Where 
the tests possess at least acceptable levels of reliabilityj this is pos- 
sible and it is^ in fact^ customary for educational tests tc* ser\re both 
purposes (Ebel^ I968). 

However, although consideration will be given to estimates of relia- 
bility and predictive validity for the evaluation measures under consider- 
ation, it can be self-defeating to force such issues too fai' in expectations 
for relatively high reliability (or validity) coefficients at the stage of 
test development represented by the present study. One reason for caution 
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in dealing v:itV, r':'liab.i.Iity i.-:; a practical paradox in requirements lor 
reliability of evaluation instraments that are intended to be sensitive to 
individual charge over time. If high reliability implies test-retest 
s tability over any extended period, then a highly reliable measure of some 
particular charaxteristic might be useless for a program that intends to 
effect individus.l change ovei' a period of months and to evaluate such 
change. Secondly, there is concern with the issue of reliability attenua' 
tion resulting v'rcm the previously cited group -homogeneity for lov; SFS or 
cultiij:'ally deprived. P'inaJly, there is the possible effect on reliability 
that stems from constraints on test length. Ibe desirabil5.ty of shortened 
test length foil ?v/s from the prior discussions of test -taking problems 
among disadvantaged adolescents. Iberefore;, unusual caution is needed 
before applying any stringent reliability demands in the choice of mea- 
sures intended for the assessment p'urposes and the particular population 
to be evaluated here. 

IT. DhViUlFMKiiT OF I'iH:: VPvLIMrNAF/ Mv\SUF£S 
/V , Genera l Charr i c t e r i sties 

On the basis of those performance objectives that can be defined for 
the Neighborhood fouth Corps Hograjn. the characteristics of thr enroJlccs 
v:ho are to be evabcatid aivd the admittedly sketchy knov.-lcdge that exist.' or 
I>robiems of assessing the capabilities of disadvantaged adolescents. is 
possible to dcl'iro some general ground rules that should be met by a bat - 
tery of relevant ^-valuation rv^asures; as well as their specific conterP. . 

At the outset, it should be evident that any measures dealt v;ith, at tl'iis 
T’hase of devclopm(Mt, cannot be- co?'icidercd as more than preliminary in 
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natm*e. They can certainly not be construed as meeting the requirements of 
foi^nal instruments. 

For the battery of 13 enrollee measures prepared for use in this study^ 
the following guidelines were applied: 

(l) Format : Test items are printed in small booklets (approximately 

X 8") with the content for each item generally appearing on only a single 
page. (For about 25^ of the test material there are as many as three items 
spread oyer two facing pages of the booklet. ) Seven of the 13 test booklets 
contain pictorial information^ accompanied by supplementary verbal material 
below ; or within^ the pictorial scene. The intent is to communicate as much 
information as possible by not relying exclusively upon either a pictorial 
or verbal mode. Boxes to be marked by the respondent are next to each item- 
response choice on the page* (A glance at the sample items in the pages of 
this section provide an overview of the format. ) Verbal directions which 
serve as a capsule summary of the longer^ orally presented directions are 
printed on the front cover of each test booklet^ where space is also pro- 
vided for the individual to write his name and TI.Y.C. project. No test 
titles appear on the cover page^ or anywhere else in the booklet* Separate 
male and female foms are required for the seven measures that utilize 
pictorial information^ but the format always remains the same despite appro- 
priate content changes. 

This format requires a considerable amount of art work^ more complex 
printing than is customary^ and hand scoring of items. Thus^ it is appar- 
ent that aspects of logistical convenience in test development and usage 
were intentionally ignored for this phase of study. 
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. .. (2). -. Verbal Content ; Printed material (^^hich is also orally presented) 

was maintained at a relatively low verbal reading level. Approximately a 
f if til grade reading level was chosen and verified by a check of the test 
booklet having the most difficnJLt reading level against the Thorndike- Lor ge 
Word List (Thorndike & Lorge^ 194-4). (However, an estimate of comprehension 
level for orally presented material, that is also printed, cannot be made 
with any precision.) Directions, both oral and printed, are kept at a 
similarly low verbal level. Slang expressions, generally suitable for 
adolescents, are used where they seem to fit“-but not stressed unnecessarily 
in the overall verbal style. Verbal content and concepts are, insofar as 
possible, relevant to and understood by adolescents from a disadvantaged or 
low SES background. 

(5) Pictorial Content ; All pictorial scenes depict a youthful (adoles- 
cent) figure in some situation that portrays the item intent for the given 
measure. Thus, clarity of pictorial content is critical and simple black- 
line drawings (i-e., "cartoon” style) were utilized. Since verbal material 
is coupled with the pictorial material, the information in either mode tends 
to be supplementary so that ambiguity can be minimized. 

Any choice of pictorial material to accompany the verbal was always made 
on the basis of its providing some degree of supplemental information and not 
merely for uniformity or for its own sake. Pictorial information is also 
intended to provide some degree of added interest to the test content and, 
hopefully, to enhance motivation (Weingarten, 1958)- If overdone, however, 
with little apparent relevance, its use can appear somewhat demeaning to an 
adolescent — despite his low verbal reading level. Where the meas\ires are to 
be applicable to a number of racial or ethnic groups and the respondent is to 
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identify vith the youthful figure in a pictorial scene ^ it seems logical not 
to portray any particular racial characteristics. Hcr^ever^ this is rather 
diffici-ilt for an artist to portray and the choice vas made to “neutralize” 

(or at least minimize) any feat'OJ'es that depict race by specifying simple 
line ii’awings of the type that are found in the Rosenzveig Pict'ure Frustia- 
tion Test (Rosenzweig, 1948)= The scenes depicted are intended to possess 
some degree of situational familiarity to the individual. Although this is 
difficult to accomplish in picturing a number of job-tasks and job-related 
scenes ; many of th^. pictorial scenes are probably part jf a broad enough 
fi'ame of cultui'al reference on the basis of their appearence in printed 
media, in movies, or on televisicn. 

(4) Administration ; The measures are intended foi’ administration to 
small groups (not exceeding about 12 iiidividual s per group), v.lth separate 
sessions for testing males and fcralcs. Drrections as well as a.l^ item 
stems and response cVioiccs orally Tf'csented by the examiner, "^hus, 
there is a specific intention to minimize the influence of any verbal read- 
i»ig component on each measurr;. As will be apparent in tne detailed descrip- 
tien of the moasujes, a smll poiporlion of the items do contain verbal 
material accompanying the itcrri stcTi and rmsporsse choices (e.g. ,. a nevrspa]-- r 
want ad) that could not logically be read by the cxami!;ci’ without ircriCdi- 
ately revealing the correct iteji resporses. 

Test Length : Since the “test-taklng-atte ih iC'n -sTan" of the- in- 

tended respondents can oc expected to be folatively short, it is pru.lent tc 
maintain test length at the rrdninum thought Ticccssary to cover* a dc-sirtd 
dimension, ani to do so with a sufficient range of content to result in a 
sor.evhat reliable measur e. In additio:,, at the prtnent stage of d^: velL'}'i:m n1. , 
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and for the relatively imique approach to test material and format used here, 
it -would he wasteful to construct numerous items for any measure in which all 
the items of a given type might prove useless. 



B. Description of the Measures 

The 15 preliminary evaluation measures for use with male and female 
enrollees are to he described in this section, along with two criterion rat- 
ing scales used hy N.Y»C. counselors and work supervisors in evaluating the 
enrollee. Brief discussions of the content, format and scoring, as well as 
some aspects of the rationale, are presented along with samples of items from 
each measure. This material is summarized from more detailed discussions of 
these areas, available in the previously cited document on specifications for 
the measures (Educational Testing Service, 1967)- 
Job Knowledge (JK) 

This measure, consisting of 27 items for the female form and 50 ibems in 
the male form, is intended to assess the degree to which the enrollee is 
aware of a number of categories of job requirements. For each form there 
are three multiple choice questions asked. These relate to an occupation 
pictured, along with its corresponding job title printed below. In the 30- 
item male form there are five items covering each of the six job knowledge 
categories — (l) education required, (2) where the work is performed, 

(5) full-time starting salary, (4) the specific major task performed, 

(5) the most likely working hours required and (6) tools required (with 
response choices consisting of pictured tools). The female form contains 
only the first five knowledge categories since female occupations for which 
tools can be identified are difficult to uncover. 

No published, formal, measure is known to exist in the area of job 
knowledge that would be suitable for this battery. The "Vocational Interest 
and Sophistication Assessment" (Parnicky, Kahn, & Burdett, I963) that has 
been under development for use with individuals having low verbal reading 
levels is used to obtain scores on both "knowledge of job conditions" and 
vocational interest." But it is exclusively pictorial and was designed for 
use specifically with educable mentally retarded. Thus, although its format 
and rationale for pictorial material served as a valuable aid, its direct 
application to a culturally-deprived adolescent group was considered ques- 
tionable . 
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Scoring of the Job Knowlectge measure is based upon total, number of cor- 
rect responses obtained from the four -cht. ice, multiple -choice , items and can 
also be broken down into subscores based upon the nuniber of correct responses 
for each of the Jcb Knowledge categories* 



Figure lA 




CASHIER 



W*i«ra would /on wrtr on this 

Job oost. of the tine? 

□ In I f«ctor7 on the 
line 

□ la the kitf-hi'r of a res- 
taurant 

J In i store 
n fOUT om hone 



0 letting 



Figure IB 




MAH. CARRIER 



Atout how aruch would /ou jet paid 
wMa /O'l start on this Job full -tine? 
r~| *2. 25 to P. VS an hour 

[ I 5C< to 7tt an hour 
( [ ll.CW to 11. JO *» hour 

f ~] 13.50 to «r hour 
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Vocational Pla ns and Vocational Aspirations (VA, VP) 

These two measures are appropriately considered together since they uti- 
lise the same stim'olus materials (although d5fferent response choices) and 
are also scored in relation to one another* Each booklet contains, in both 
male and female forms, l 6 items in the format shown below with pictorial and 
verbal material for each item on a single page. The items were chosen on the 
basis of their social status levels. Status values were derived from the NORC 
occupational prestige rating studies (Dimcan^ Hatt, & r^orth, I 961 ) and the 16 
items have been ^ 'ouped i? *:o eight statu levels based upon these values, with 
two items at each level. 

The individual responds to each occupation shown on a four -point scale 
that indicates the degree to which he plans (or aspires) to that occupation. 

TJiere are no foiTual pub.lcLshed scales for measurement of vocational plans or 
aspirations, as intended for use in this study. IVo published rneasm'es which 
deal viuh occupational stimulus materials are, in fact, used for personality 
assessment purposes (Ammons, Butler, & Herzig, 19^9; Holland, 1959). Highly in- 
formal check lists and questionnaires, or simply open-ended questions, have been 
used in research with ado?_escents or youn^ adults ' (Empey, Ginzberg, Ginsburt 

A>:elrad, & Herma, I 95 IJ Slotkin & Forlano, I 962 ; Stephenson, 1955; Varo, 196 ^ 1 ). 
Rut the nature of their content, format, scoring and lack of measurement proper- 
ties would make them inappropriate as applicable formal instruments. 

Scoring for the VA and VP meas'ures c -insists of a ’’status discrepancy” 
score for each one, based vbon the difference between the -weights assigned by 
the respondent to each status pair. (For example, if the pair of items at 
status level eight (&)- -’’physician” and ’'scientist” --‘were assigned degrees of 
choice ”4” and ” 2 ” respectively, the status discrepancy at that status .level 
is a ”1.") The total discrepai;oy score is thus the absolute sum of these 
individuail values for all items hi the measure. In effect, f'jis is inton led 
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as a measure of the consistency with which the individual uses an 'underlying -. ^ 
status continuum for his vocational choices. A second score is possible by 
utilizing the difference between the level (i.e.^ degree) of choice assigned 
an occupation in terms of plans and aspirations. For example, “Auto Mechanic” 
may be assigned a high degree of aspiration with a weight of “4“ but only a -.r 
moderate weight (e.g., a ”2") for the plans measure. The difference score is •' 
thus the difference in these weights assigned by the respondent and summed . - 
over the corresponding occupational pairs in each of the two measures. This 
difference between plans and aspirations for the same occupations is felt to 
have possible uses as a measure of the degree of "reality” in vocational 
choice — based upon what the enroUee would like to do vocationally and what 
he expects to do. • - 



Vocational Aspirations 



Figure 2A 



Figure 2B 




HOSPITAL ATTEKOANT 
Helps take core of sick people 
in 0 hospitol. 



□ A (rest Job; the kind of thing I 
vould like. 



□ It'i. pretty good; I wouldn't *lnd 
doing thii. 



□ Hot the greatest >I'd do this 1>" 
there weren't anything better. 



□ This Is a rotten kind of Job. I 
wouldn't like It at all. 
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DOCTOR 

Examines people and tells them 
what ta do when theyVe sick. 



□ A great Job-, the kind of thing t 
would like. 



□ It's pretty good; I wouldn't Blind 
doing this. 



□ Hot the greatest -T*d do this If 
there weren't anything better. 



□ This is a rotten kind of Job. I 
wouldn't like It at all. 



C opy r 1 gni. © by Euui'at I ^ na 1 Tes 1 1 ng Sicrv 1 c e 



Vocational Plans 



Figure 2C 



Figure 2D 




HOSPITAL ATTENDANT 



□ The oddfl are I'll do scmethlng 
like thla. 



□ 



There's a fair chance I would wind 
up in this Job. 



□ 



Maybe r would try thU, hut I 
don't think the odds are too good. 



□ 



This is vny out for oe. I 
wouIA'i'l <sven expect to look 
I'cr this kind of Job. 




DOCTDR 



□ The odds are 1*11 do sooetMng 
like this. 

□ There's s fair chance 1 would wind 
up In this Job. 



□ Maybe I would try this , but I 

don't think the odds are too good. 



□ This is Way out for ae. I 
wouldn't even expect to look 
for this kind of Job. 



Helps take care af sick people 

in Q hospital. Ccpj-rlght © by Educutinnal Testing Service 



Examines people and lells them 

what to do when (heyVe sick. Copyright © n-f.7 by VViucaUcnal Te:.tinp Gervi.*e 
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Interest in Vocational Tasks (VI ) 

As the title implies^ the attempt here is to measirre an interest pattern 
based exclusively upon specific occupational tasks, rather than interest 
patterns derived from choices based upon occupational titles or from the more 
customary combination found in formal interest measures of leisure time activ- 
ities (i«e., hobbies) and vocational tasks- 

Criticisms involving verbal level, test length, t;echniques of administra- 
tion and cultural relevance of content can be made of the widely used Strong 
Vocational Interest Blank (Strong, 19 ^ 5 ) and the Kuder Preference Record 
(Kuder, 1956 ) when these measures are applied to a disadvantaged group. 

Although it was developed for men in nonprofessional occupations (i.e., most 
skilled and semiskilled), similar criticisms can be aimed at the liLnnesota 
Vocational Interest Inventory (Clark & Campbell, 1965)- Some of these criti- 
cisms have been touched upon by Clark (I96I) and Barnette and McCall (196^^).* 

In contrast to these verbal measures, there are two exclusive}y pictor- 
ial interest tests (Geist, 1964 ; Keingarten, 1958 ) which also possess short- 
comings for use with disadvantaged or minority groups. The level of vocational 
tasks portrayed, the clarity of the pictorial material, the clearly Caucasian 
figures shovoi^ and the standardization data can serve to limit their applica- 
bility. One of these measures, the Geist Picture Interest Inventory (Ceist, 
1964), has also been criticized on the basis of shortcomings in its development 
(Shimberg, I969). 

The approach to interest measiu'ement that proved helpful in the design of 
the present measure is the l68-item, all verbal, California Occupational 
Fi'eference Survey (Knap (ir^tnt, & Demos, I966), Obviously, it is not a 
measure that meets tliC ground niles for direct application to the adolescent 
group of interest in t-ras study, }icr^ever, its teclnical ap>proach and content 
serve as a valuable model, since it utilizes only specific occupational tasks 
extracted from the Dictionary of Occupational Titles (USES, I965) anid it scores 
t}]G degree of interest in tliese tasks for occupational groupings or clusters 
that are derived from factor analyses of the items. By utilizing those items 
that could be pictorializcl reasonably and those occupational groupings in 
which these items fell (along with some rewording of tasks where verbal level 
was considered too hi g}]), a P8-itcrn moasui'C was construe tei with seven occupa- 
tional clusters containing foiu items each, Ihe seven clusters axe termed: 
Clerical, Service, T<’c}ancal, Business, ^ratdoor. Science, and Aesthetic. 

Scoring consists of su:ri:'’ng t}ie weiglits assigned to the vocational tasks 
(1 to 4 ^ along a "favorabJ -rni’avorablc'' contiuuu:;i) for the group of tasks in 
each of the seven occujati rial clusters- 
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Pigure 3A 



Figijue 5B 




□ 

□ 

□ 

□ 



Pr«tty V>»ia - X couldn't t«ki toln« It. 

Bat too food - 1 mlflit do U>1^ but I 
vould&'t lUt lt> 



tM# li OK - I wouldn't *lod tbl* 
too »vicb. 



Thl# 1# |r*»t - Joit th* bind of thUf 
I vould lU* to do. 
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Attitude Toward Auth orit y (AA) 



This attitudinal measure consists of items in the female form and 
15 in the male form. The single item on a page always depicts an adoles- 
cent figui'e being spoken to by an authority figure (i-e.^ judge, supervisor, 
teacher, policeman, father, mother, truant officer, job intei^viever ). ilie 
response given by the adolescent to the authority figure is either a clearly 
anti- or pro-authority response. Item stems are always in the saine form as 
shown in the two sample items below--i.e., the number of people who you think 
would give that answer in this situation — with the five item choices covering 
some range between the odds of ’^1 out of 10*' to *^9 of 10.” The technique 
used here has come to be kncrwti generally as the "disguised factual” approach 
in testing and has been applied successfully to adults from a wide range of 
SES levels by Bernberg (l95^i)- In his manual for the Himan Relations 
Inventory (}{Rl), Bernberg refers to this item style as the "direction of 
perception” technique and he has used it for measuring social conformity with 
the all-verbal }iRI. Two other formal measures related to the confor- 

mity area, and usable with young adults, are the Socialization Scale of the 
California Personality Inventory (Gough, 1957? 19^0) and the Family Opinion 
Survey (Elias, 195^) that measures attitude toward authority re.lationships 
w”ithin the family. Tnese tVa*ec formal measiu'es ■v;ere considered too narrow in 
content (and validation) and too highly verbal to meet the intention of the 
present measure, or to be adopted directly. 



Scoring of the AA items is based upon the assigiiment of weiglits 1 to 5 
along a continuum of choices with a weight of 5 repi-esenting the high, or 
l-)roauthority, end of the scale. Thus, for example if the item depicts an 
antiauthority response made by the youth to an authority figure, a response 
choice of ”l out of 10 p>eople” would receive a weight of *’5” (the respondent 
is proauthority and thinks that few people would rake such an antiauthority 
remark); w'hile a response at the other end of the continuum (”5 out of 10 
people”) on that same item represents an antiauthiorxty attitude arid receives 
a weight of ”1.” Based upon Bernberg findings, it makes no difference in 
the overall score whether the response -choice continuiui on ajiy item ranges in 
steps from, say, out of 10 people” to ”5 out of 10 people” or from ”9 out 
of 10 people” to "9 out of 10 people.” It is only tl'ie I'clative location on 
the continuum that deteriiines the contribution of the res] onsc. 
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Fii,^u : 4 a 



Figure 4 b 




* i'll do tr«y bolt to boch on 
limo ^rom oow on* 



Studiei ili<7V that how Bao^ 
{xopla five that aniwer to the 
hoaiT 



□ 

□ 

□ 

□ 

□ 



5 out of 10 j>eopl,‘ 

6 ojt of ID t^ople 

7 out of 10 people 
fl Out cf 10 penjile 
9 out of 10 people 






L\ ■ L 




*r« *n*l loom tm ■ 

"o;>Tl^V.t 



Mow Mn/ itudeiti 4o pou think 
wovdl |lv« thit mivc.T to • 
taichtrT 



□ > 
□ > 



out of ].o atudautt 
out of ;.o itu^nta 
O'jt cf '..0 itufljnti 
out of -0 ituierta 
out Of ;.o iti'&eotB 



© l?^"’ hy L^<’:it I ':r«l Tetllrg Ter/ 



S elf-Esteem (SE) 

"Hie l 6 items that constitute this measure consist of pictorial scenes 
in which an adoles ^nt is portrayed in ■various academic^ social, and euiploy- 
ment settings a The respondent is asked to imagine that the young person in 
the picture represents himself. The response rt<luired is one involving; his 
estimate of what would happen if he were, in fact, the youthful individual 
in that scene and is intended to reflect his level of self-worth or self- 
esteem on a ^ -point scale. 

Each item (and the aggregate of items) ore intended to deal with a 
dimension often conceived of as competence in interpersonal social acticn- 
Wylie ( 1961 )^ vho reviews nruch of the research literature in this area r:.nd. 
the different approaches to quantifying self-esteem, notes the general lack 
of validity data to support the variety of definitions and measures used in 
dealing with the concept. As her review indicates, most self-esteem measures 
are informal, exclusively verbal and involve a fairly complex clinical ration- 
ale in their interpretation and scoring. One such informal scale that lias 
been utilized on reasonable size samples of adolescents (but of \mquesti onably 
higher academic levels than an N.Y.C. dropout group) is a i\.rely verbal 10- 
item measure that lacks adequate analysis of its psychometric characteristics; 
with item wording that is questionable for the N.Y.C. enrollee group (Rosenberg 

1965 ). 

Although two published fonnal measures exist that provide some form cf 
self-regard score (Gough, 19?f; Rogers, I 96 I), there is no evidence of their 
suitability for a disadvantaged adolescent group. In addition, one of these •- 
the Rogers "Personal Adjustment Inventory" (Rogers, 196l)--is intended for 
grade school children 9 to 15 years of age and its content is inappropriate 
for adaptation to older adolescents. The other measure is questionable for 
direct app'>licatlon in this study because of its verbal level and its general 
cultural content. 

Sc orirn^ of tho total SK measxu'e consists of sumioitig alJl item weights 
wliere the weights from 1 to 3 on each iter: represent a low-to-high continuum 
of self-es l-eem. 
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Figxire 5A 



Figure 5B 



' 





□ 



^rottbl/ aothli 
could do for ■«! > 



□ 



I know I could K&ke 
If he hel;«d a< set 
trtlBjr.g. 



□ 



I ktun-1 chu: 

If I co-^d get 9?Dc 
t rate log. 



Ceryr t 0 '~y I'd 



Deferred ^ratification (dg) 

The ability to delay present rewards for future gain (i.e.^ defer 
gratification) represents the intended attituiinal dimension for this 16- 
item scale. The booklet for this measure contains a total of 20 items, four 
of which are common slogans that serve as distractor items and are unscored. 

The deferment of impulse gratification as a behavioral concept was 
introduced into the sociological literature by Davis and Dollard (l940) in 
an attempt to deal with patterns of behavior that differentiated the middle 
and lower social clashes. More recent research dealing with measurement of 
this concept has been made by Schneider and Lysgaard (1955) and Straus (I962) 
who have attempted to quantify a "deferred gratification pattern" (DGP). 

Their res-jlts led to several scales that may have value for defining impor- 
tant predictive responses in adolescents. Straus (19^^)^ for example, found 
modest correlations between the DGP and measures of academic achievement and 
occupational aspiration; even with intelligence and SES partialed out. Thus, 
this type of measure may be capable of meaning foLV differentiating between 
individuals in the adolescent group that i^ utilised in the present study. 

The subscales with the greatest utility, based upon the prior research, 
appear to be those defined as; "Hast> Aggression" (e.g., "If a guy gets you 
sore, hit him- -no sense in waiting until you Cra]m down"); "Freedom of Spend- 
ing" (e.g., "Money is to spend, not to save") and "Affiliation" (e.g., "You 
have to be cai’eful about the kind of people you pick for friends"). A fourth 
subscale was derived from the more recent attempts to deal with gratification 
deferment in tenns of specific delay of reinforcement as used in research 
with children (MLschel, I961). This sub scale is termed "Specific Delay of 
Rew'ard" (e.g*, "l would rather have a cheaper used car today, than save up 
imtil I could afford a new one") and is not one that has been used in the 
past as part of the DGP scales. Item-response format is similar to the dis- 
guised factual approach in that the individual responds on the basis of the 
degree to which he thinks other people uiiglit agree or disagree with the 
statement (5 -point scale). 
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Scoring consists of the sum of the weighted scores over all items, with 
a high score assigned to a high level of deferred gratificati( n. 

Figure 6 a Figure 6 b 



] VOVUi TELL A COP CfT If HE XE - 

ivof IP IT KEJurr omi«3 m TPoatLF, 

^ "j nolKMly woiild feel thlr 

|~~j Onl/ ■ few people "IsHt V!»^* 

□ 

□ 

□ 



Sc»e people would *«ree witH thlfi 
A lot of people would feel iMs vay ■ 
Hoiii ever/b.>ly would a^ree vltfi 



■ T/fle,'. 



o 



Bj'T wiATEvifl i3f; wAjrr, thepx's aimaks e w: vai ro 
FM IT LFF. 

[ ] nobody would feel iMa vay, 

Pnly a few people algU a«ree witi thla. 

[~~ | -''*■* people would a^rer witH thli. 

n * people would feel this wa/. 

□ ^^o»t e k'e rj ' jo dy would aeree with l^ia> 



J ob Seeking and Job J f olding Skill s ( JS ; Jll ) 

niis measure is an attempt to deal ■^vith what are probably the most 
obvious areas of behavior and knowledge that it is feat an enrC>J.ee should 
develop in a youth-work training program--iae. , knowledge of what to do to 
find a job and, having done so, how to behave in order to retain it. Hie 
items used in the measure represent a direct translation of those areas of 
job-holding ain job-seeking skills that can be defined reasonably, and are 
based upon the rather limited knowledge of just what these specific skills 
or behaviors might be. The limitations stem from a glaring lack of meaning- 
ful research concerning reasons for hiring and dismissal of employees and 
what an employer uses, as the most relevant considerations, to make such 
judgments , (Reasons that often I'ange from outright bi3s to '^pet" annoyances 
or iun-/ritten x^crsonncl policies, are not readily obtained irom personnel 
files of a compaiiy. ) 



I^evertlieless, some broad areas of agreement are possible on tlie most 
iiriportant areas of job -related behaviors and can be fouiid in coui*ses that 
have been dravm vip for the purpose of teaching such skills (Educational 
Design Inc., U. S. Office of Economic Opportunity, I906) or from the 

"herw to" books or pajriphlets that deal with job success (Feingold & List, 
1998; Magoun, 1959; U* S. Dept, of Labor, 196^). No published foriiial 
m.easures of these job -behavior skills Irave been found in the test literatm'c 



Ihe first portion of the JS ; JJi measui’e consists of ly items in a foiu’- 
choice, multiple -choice foni^at, that deal vrith job seekiiig skills (JE). 
Questions are concerned with v:here to look for a job, the contents of 
several v;ant-ads, that are reproduced in the booklet, and questions about 
paints of application blanks, llie respondent is required to choose the 
correct ansv.'er for each itemi TIjc sccon > section contains 11 itemis dealing 
v;ith cn-the-job (job-holding) beljavior (dif), covcri?\g t}.e customary ccnccrns 
witj'i j'UJ'ictual ity, a}iropriate dress, rosp^nco to a sinoi’visor ' s ^'cquest and 
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some items dealing with what are probably less -often revealed soiirces of 
employer discontent, snch as placing bets and drinking on the job. The 
respondent chooses one of three response statements that describes how he 
feels — or what he iroiild say, or do, in the particulax- situation shown. 

These responses are scaled *^1*' to "5" on a "low^' to "high" (least correct 
to most correct) job -holding skills continuum. 

Scoring for the I 7 items of the job-seeking skills portion consists of 
the number of corr^ict responses. For the 11 job-holding items, the response 
weights 1 to 5 on each item are summed over all 11 items. The two scores 
are always treated separately for purposes of analysis. 

Job Seeking 

Figure ^A Figure 7B 



permanent Selling 
PosiHoru in 

iMOYTOWfAl 

COMEna 

Futia 

WOMENS SHOtJ 
mSJtlY 
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NO NIGHTS 

Exce^Icnr Bantfih 
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Eirpl'04^. lirt* fh 

Ford 6 Baylor 
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When would you vork on this Job? 

□ 



At nightB only for five days 
a week. 



n Only on Thursday, Friday and 
Saturday . 

[ I Only during the 

1 " [ Any hours you want to pick* 
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If you've never been sarrl«d before which 
one I'Quld you oorX on line nuaoer 6? 



I I Single 
( [ Divorced 

I I Married 
( I Widowed 

What would you put on lino nunber 2 where 
it Says '’present address?" 



□ 

□ 

□ 

□ 



The address of the place where 
you Jive now. 

The address where a friend lives. 

The address of the place where 
you're applying tor the Job. 

The address of the place where 
you worked lost . 



If you have an uncle who works for this 
place where you would like a Job and 
your uncle told you about cooing here to 
apply for it, where would you write your 
uncle's nace? 

I [ Li ne 1 

I I Line 8 

n ^ 



Ccpyrlgh’ Q i'/jT vy Kvuuati.,r,a; Testir;,: Der.-lce 
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Job Holding 



Figure 7C 



Figure 7D 




Vhiit vould you say to a boss who 
said this to you? 



□ 

□ 

□ 



Sorry, I'll try to here on 
tine frciii now on. 



Its tough ror ne to make it 
here on tine but I'll try. 



I do ny work - what's the 
difl’ercnce if I'm late. 



If you feel too alck to go to work one day, call the bosa and let 
hln know. 

I I I wouldn't bother - It wouldn't nake any difference to hln. 

I I I might call If I h^pened to think of It. 

I I I would alwaya call - otherwieo he'd get aore. 
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Motivation for Vocational Achievement (MVA) 

Seventeen items ^ all verbal^ are set up in a 5”Point Likert-Scale 
format^ as shown in the sample items below. The first 10 items consist of 
the item statements followed by the 5"Point' scale representing degree of 
applicability. (This format has proven successful for use with adolescents 
in a formal scale known as the Demos D-Scale (Demos ^ 19^5) which is intended 
to identify potential school dropouts.) For the next seven items of the 
measure^ a somewhat different format is used that was developed for assess- 
ing adolescent work attitudes in a youth-work training program (MPY^ 196^). 
The item responses are conditional situations regarding employment and the 
5-point scale represents the intensity of willingness to stay on the job. 
(Would you stay with it if: "The work was dirty?"; "You had to work long 

hours?") There are no formal measures known to exist in the specific area 
of vocational achievement. However ^ a variety of primarily informal scales 
that have been used for measuring "achievement orientation" are described 
by Kahl (1965). 



Scoring for each of the two sections of MVA^ with their somewhat dif- 
ferent formats^ is based upon the sum of the scaled responses^ with the 
higher value representing the higher level of motivation. The two scores 
which are designated MVAI for the first 10 items and MVAII for the remain- 
ing seven^ are treated separately throughout the analyses. 

Figure 8 a Figure 8 b 



IT PAYS TO TRY TO MAKE SOMETH IHG OF YOURSELF 
BY WORKING. 

I I Hearty almya 
[~~| Moat of the time 
[~~| Sometlaea 
{ { Very few timea 

[~~| Almost never 



Would you atay with it if: 

niE BOSS WAS ALWAYS OH YOUR BACK? 



□ 

□ 

□ 

□ 

□ 



X vould alvaya atay with the Job anyway. 

Moat of the time Z would stay with it. 

Saaetlaes I might stay with the Job. 

I probably wouldn't stick it out. 

Z would never stay with the Job if this ves 
the 'fay It vaa . 
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Practical Reasoning- Reading (PRJ>l) 

This measiire and the tvo succeeding ones probably comprise the closest 
approach to intellectual or cognitive measures in the battery. The ration- 
ale for their construction and use is based upon the assumption that^ within 
a work setting^ there are certain forms of skill for which an enroUee must 
demonstrate at least a mininum capability. These skills often center around 
following directions required to carry out some aspect of the job successfully. 
The intent here 'is to arrive at several indices of 'Veasoning^^ ability by 
utilizing material which presents problems germane to the practical work 
situation. It is also assumed that the practical reasoning measxires represent 
dimensions of ability that are somewhat independent of conventional intellec- 
tual measures since they are not as highly dependent upon verbal reading 
skills. The 1^-items in a fouj;-choicej multiple -choice ; format that consti- 
tute the Map-Reading measiire^ are based upon questions relating to an assumed 
job that requires the respondent to make deliveries in an area represented on 
a map (shown below with a sample item). 

Scoring consists of the total number of items answered correctly. 



Figure 9A 



Figure 9B 
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Practical Reasoning — Zip Coding ( PR2 ) 

In this 10-item measui^e the responses to the multiple -choice iteizs are 
based upon the respundent’s ability to "sort'" mil (in an assuuied post office 
job) based upon zip-code t;^^^ of information. Tne ai'bitraril^^ designed zip- 
code chart and a sarple item are shcr’;,7i beltm>:. 



Scoring consists of the total number of items answered correctly. 







Figure lOA 



Figure lOB 
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Zip Codes 



Area of 




SUtB 
Hew Toric 


Zln Code 


City 

Jamaica 


Zio Code 


th. country 
Horthoast 


1 


Haibere 

40 


Ninbers 
10 to 25 


Pertnajlyania 


50 


Elkins Park 


20 to 30 


Mldwost 


6 


Kuiau 


65 


Lawrence 


45 to 50 






Hlaeaurl 


70 


St, Louis 


eo to 94 


Kest 


. 9 


Vaahington 


94 


Seattle 


60 to 79 






Oregon 


79 


Eugene 


70 to 74 






Callfomia 


69 


Burbank 


50 to 55 



Copyright (c) 1967 by Educational Testing Servl: 



You vould kncrv that a letter vlth the zip code amber 9T953 should 
go to vhat geaeral srea of the countiyY 

I I Mortheaat 

□ 

I I South 
I I Hldvest 

Copyright © 1967 by Educational Testing Ser/1 



Practical Reasoning — File Card Sorting (PRP) 

This is also a 10-item, imiltiple -choice, measure. Attached to the test 
booklet is an envelope containing eight numbered file cards that represent 
information about job applicants. The respondent holds an assumed position 
in an employment office and is asked by his boss to sort out individual 
applicants based upon characteristics given on the cards, so that these men 
and “women can be considered for various jobs as described in each item. 

(a sample card and item are shown below. ) 

Scoring consists of the total number of items answered correctly. 

Figure UA Figure UB 



Kra. Cook, Angola 
1A3 - U9th Street 
Flushing, Long Island 
Hew York 



Dat« Filod: January 2. 1967 

Age; 22 _ 



Harried ; Yes X Ho 

Humber of school 5 6 7 8_ 

years canpletad ; 9 10 X 11 12_ 

Jobs you can do ; drive a car; good with figures 

Salary wanted ; $300 a month 



Look only nl the curds for voacn. Use the lust nunes 
rind liut theiB in ulphabotlcul order. 

Trtr* right order vuuld be: 

I I Card 3, Card 1, Cord 2 and Card U. 

I I Card 1, Curd 3, Card t and Card 2, 

I I Card 7. Card 3, Card U and Card 1,. 

r 1 Card 2, Card 4, Card 1 and Card 3. 



CopyrVht @ 19O7 by Educallonul Testing Service 



Copyright © 1V67 by Educatiofjal Tenlnr "orvtrc 



Enrollee Eating Scale (ERS ) 

The 15-item, 5“Point rating scale is intended as a measure of the ])T.Y.C. 
enrollee ’s feelings about the particular project in which he is enrolled. 

It cavers three aspects of the U.Y*C. project having to do with the work 
supervisor, the counselor and the N*Y*C* project in general. The use of this 
measure as an evaluative tool assumes the possibility of enrollee change for 
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what can be considered an important phase of his attitudes. The items are 
phrased in language understandable to the enrollees since they were collected 
from interviews with a sample of male and female enrollees at six projects. 

This material was obtained in response to a question dealing with what the 
enrolled liked or disliked about their counselor, the work supervisor, and the 
N.Y.C. project in general. Specific item statements, derivable from the inter- 
views, were placed on cards and those 15 occurring with the greatest frequency 
constituted the ones chosen for scale . 

Scoring for the scales is based upon the sum of the 1 to 5 item weights 
{•'unfavorable to favorable”) over all of the l5 items. 



Figure 12A 



Figure 12B 
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□ li c«,«r it 1*. 
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^ ^ ScattlM* thi, li ,0. 

Thi» 1* Bufli “V It li. 

□ '■ tbc va. It 1,. 



ThE C:UTIG.EU;P T>IES TO E2 K0ltK5T *JTL p^'^R VITO TOU. 
[ ] rh:i 1, Mver tb. May it U. 
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Counselor and Work Supervisor Rating Scales 

Two scales of 11 items each serve as the criterion measures for the study-*- 
one is the Counselor Rating Scale and the other the Vfcrk Supervisor Scale. Since 
these are the two adults who undoubtedly have the greatest amount of personal 
contact with the enrollee during his tenure in N.YaC., as well as the most rele- 
vant quality of contrast (i.e., in the counseling and job situations), they would 
appear to constitute the most reasonable evaluators of an enrollee •s status and 
progress at any point in the program. All of the items in the scales are shoT^ai 
in Appendix C. The material for the items was collected from interviews with 
counselors and work supervisors at six N.Y.C, projects and the items constructed 
from the interviews deal with favorable and unfavorable enrollee characteristics 
as seen by these two groups. Counselors and work supervisors were asked to make 
these evaluations independently of one another, and neither group had any knowledge 
of enrollee performance on the battery of measures. 

Scoring of each scale is based upon the sum of the 1 to ^ item ^'eights, along 
the unfavorable- to- favorable continuum, over all 11 items, ;rith a weight of ”1” 
alw’ays assigned to t })0 unfavorable end of the continuum and a ”5” to the favorable 
end . 
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III. METHOD 



A. Sample 

A total of 256 Neighborhood Youth Corps enrollees in the Out-of-School 
program constituted the sample of respondents for the study. Of these 125 
were males and 155 v;ere females* Administration of the measures was carried 
out separately for male and female groups* Projects v.^ere requested to pro- 
vide enrollees who had been in the program for about one month and no more 
than three months. Thus^ the enrollees represented a sample in the earlier 
phases of their N.Y.C. participation. This not only served as somewhat of 
a control on uniformity of the group *s characteristics^ but was intended 
primarily to provide a minimum level of counselor and work supervisor 
acquaintance with the enrollee for rating purposes^ as well as enrollee 
acquaintance with the program for his rating of N.Y.C. 

Eleven N.Y.C. projects cooperated in the study^ nine of which are 
located in Nevr York^ New Jersey^ or Pennsylvania and can be considered as 
drawing their enrollees from exclusively urban areas. The other tm proj- 
ects are located in southern rural areas — one in Maryland and the other in 
Alabama. 

Approximately 85^ of the male sample were Negro and 15^ were white or 
of Puerto Rican ethnic background. Of the female group^ approximately 90/^ 
were Negro and 10^ white or Puerto Rican. All Spanish- speaking enrollees 
in the sample were sufficiently bilingual to comprehend the material with 
a minimum of additional assistance. 

The sample cannot be construed in any way as representative of the 
national composition of the N.Y.C. Out-of -School program--not only because 
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of geographic restrictioi. but, more obviously^ since it lacks representation 
of other ethnic groups that make up the enrollee population American 

Indians Mexican -Airier icans )■ ^' llo^rever, the sam^^lo is considered adequate 
for the pui‘p""-r^s of thio initial phase of development for the evaluation 
measures. Criterion ratings vere obtained from the N.Y.C. counselors for 
110 of the 11:) male em-ollees and 119 of the female enrollees. VIork 
Supervisor ratings were miade available for Il4 of the males and 111 of the 
females . 

B . A'dmin i s t ra t i on 

An informal tryout of the items for each measure vas first conducted 
v;ith a small group of three m^ale and four female enrollees at one N.Y.C. 
project. This v;as held on a group-discussion basis and vas intended to 
check estimated tine requirements for adrriinistrationj clarity of instiuc- 
v-ions and areas ambiguous item content. Molrl fications in instructions 
and itemiSj resulting from this pretest trycut^ v;ere incorporated in the test 
booklets , 

Idle enrollee measures v.erc adrunistered at each N.Y.C. project to 
group^s ranging in sl^c from four to lu enrollees » vrith male and female 
groups tested separately. Tnese are untim-:d measures^ so that administra- 
tion is relatively unhurried. However ^ somevrhat of a built-in pace is 

*■ 

Facial or ethnic characte ristics are not utili^t ; in the data analyse, s 
of this stu'ly; nor are they considered pertinent to the study pui'pc^ses. It 
is also the preference of the project personnel that racial or ethnic in fer- 
maticn not b-' used in idc ntif icar ion of individual er.rollecs. Thus, th' 
proportions provided here arc based upon exa.miner observations and onl;/ 
intended to add to general infeuunation on sar'ple composition. r\ir t he i nor ^ 
individual enrollees are not identifiable by ctlrnic or racial characteris- 
tics in the data records. 
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imposed by virtue of the examiner having to read every test item aloud to 
the group* The first seven meas'ores were administered over a period aver- 
aging about 65 minutes^ followed by a break of some I5 to 2^3 minutes and a 
second session of approximately one hour for ijresentation of the remaining 
six measui'es* The order of administration for the 15 measures is the same 
order in which they have been described in Section 11 of this report. ' 

Directions used by the examine!’ are sho^-ai in Appendix B. As is indi- 
cated in these directions the enrollee was made aware of the fact that the 
tests had no effect upon his status in the N.Y.C. program nor was he com- 
pelled to respond to any material considered, by him_, to be inappropriate. 

Special Problems 

Certain aspects of test adrrjinistration, with a group of adolescent 
sohool-dj’opouts who are minority group members from a poverty background^ 
merit discussion and clarif ication* This is because problems can arise, 
that are not within the domain of those custoir^'ix'ily enco'ontered in the psy- 
chological group-testing situation^ w’hich apply primarily to the male en- 
rollce groups. The female enrollee s are fairly cooperative and amenable to 
standardized group testing procedures. V?ith this particular population of 
Diales, who have experienced academic or disciplinary difficulties throtigh- 
out much of their school careers, any group testing situation (no matter 
how contrived) becomes analogous to a classroom situation v;ith the examiner 
in an authority position (no ratter herw he plays the role). 

In the testing situations for this study, the primary motivation for 
enrollee participation w'as based upon thei^^ receiving the regirlar hourly 
•wage over the several hovu’s sper.t as test respondents, instead of 'working 
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at- their custoinary job sites. In effect^ there vas no more compulsion for 
an individual to take part in a test session than there is for him to remain 
cni’Ol'ed in the N.Y.C. program, a number of enrollees scheduled for 

the test sessions did not appear; others (about six individuals! simply left 
during the session. Some degree of coaxing vas 3'equired to prevent any 
large number from leaving^ particularly after the mid-break period. 

Examlner-enrollee interaction during a session, with its oral presenta- 
tion of the tests^. presents a considerably different atmosphere than encoun- 
tered in testing middle -cl ass young adults and fev; prior skills in test 
administration are applicable. Ihere is almost constant interplay that 
involves "testing" of the examiner by a variety of frustrating behaviors 
calculated to bj'ing about open hostility. Any shcru of anger in such a sit- 
uation could lead to a breaklu.m of the test session (e*g., any attempt to 
rera'imiand an individual or telling someone to leave could lead to the entii-c 
group reacting unfavorably). The sessions a] e often r^arked by a mutually 
^understood "put on" vnich is somev,dat peririheral to the test process p e i s e , 
but in vrhich both the examiner ar.i the enrollees remain av;are of the "rules 



ot‘ the game." 

The examiner’s raci.- rsay play a role in affecting the administration 
and subsequent results. i!ov;ever, it is suggested that this is less criti- 



cal than the individual examiner’s ovni m.oJc of interacting with the group 
members. (Tne point is certainly open to furtlier research consideration 
for this partirjlar group.) T'ev: rthcless . tl.ei c .remains a question of 
examiner contributi^jTi to sue}, a testir.g situation v:};ereln the cxar'U!or's 
ability to focus the respond.ents ' attentic^n on tl.e test iion.:j--and generally 
to maintain the motivation nee ded for consistent restoiiso --cannot a.Tv;ays Ve 




assuu.'.ed. 
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An additional special problem worth recognising is that the adequacy of 
a test may be dependent upon the capability of items to communicate meaning- 
fully with some intended group; but communication is a two-way street. 
Although the items of the present evaluation test battery may be understand- 
able to an N.Y-C. enrollee^ they may not always be responded to in a way 
that reflects such comprehension (i.e.j there may be a lack of willingness 
by the enrollee to provide consistent or meaningful responses). I^Jhether or 
not this is so can only be answered^, to at least some extent ^ in the pattern 
of results to be considered in the next section (Section IV) of this study. 

C. Data Analyses 

The approach to data analysis is^ of course^ tailored to the ma^jor pur- 
poses of this study. In essence^ results derived f'rom data obtained with 
the battery of preliminary evaluation measures are to be used to select the 
most acceptable test material^ to suggest modifications to tlje existin.g 
measures and to recommend further development requirements. 

The first phase of the analysis is devoted to determining item charac- 
teristics for each measure. This includes item means and variances^, item 
intercorrelations (where such intercorrelations are useful) and the correla- 
tion of items with total-test scores. 

Decisions for retaining or dropping the obviously unusable items (and/or 
complete measures) can be made at this level and the modified test material 
incorporated in the subsequent analyses. At the same time^ it is possible 
to determine whether the rural enrollee sample (constituting 2yfo of the 
total) was sufficiently similar to the urban sample in its responses to 
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aliow the data from the two groups to be combined without serious attenua- 
tion of test validity or reliability.* 

The second phase of analysis includes (a) determination of the validity 
of each measure against the counselor and work super^v^isor criterion rating 
scores as well as an analysis of these criterion ratings themselves^ 

(b) estimation of reliability in the form of internal consistency--f or each 
enrollee measure^ and (c) interrelationships between the measures of the 
battery as well as an analysis of the underlying dimensions^ or factors^ 
that make up the enrollee measures- 

Various specific analyses of the individual measures are also of inter- 
est and can allow for confirmation of prior research results (eTg._, in the 
area of vocational choice) or provide additional findings and worthwhile 
research hypotheses relevant to a disadvantaged adolescent group. 

Missing data occurred throughout the sample- In some few cases this 
occurred for complete booklets but^ primarily^ these failures to respond 
were for individual items ^ so that missing-data analyses had to be applied^ 

IV. RESULTS AND DISCUSSION 

This section presents study results ; jfirst in terms of the overall 
findings with respect to the item data^ the comparison of the rural and 
urban subsamples ^ validity and reliability of the enrollee measures and the 
characteristics of the criterion rating scales. The two final segments of 

* 

A desire to improve the representativeness of the enrollee sample 
prompted inclusion of the rui*al group as well as some assumptions by N-Y-C* 
headquarters personnel about differences in test response for rural and 
urban groups. T-TUs^ it is necessary to consider the pattern of responses 
for each group separately and the differences between them. 

•S2 
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the section are devoted to the meaning of the pattern of intercorrelations 
between the various measures of the battery and more detailed examination 
of the meaning of the results for each of the specific enrollee measures. 



A. Item Analyses 

Item means and variances were computed for each of the 15 enrollee 
measures^ along with the item-total test inter correlations for the rural 
and urban samples (within the male and female subgroups separately). This 
four-way breakdown of the item data is presented for all items in Appendix 
A. The intention is to consider the overall pattern of item suitability 
for urban and rural groups and to eliminate items which show characteristics 
in both groups of: (a) little or no variance^ (b) overly high item means 

(or too easy in terms of difficulty levels for those items scored on a 
"right-wrong" basis) and (c) lack of positive item-total test intercorrela- 
tions indicating their failure to contribute to reliability of the overall 
measure.* 

In general^ the approach to item or test elimination was highly con- 
se^rv'ative because of the preliminary nature of the measures at this phase 
of study. So that only items which were particularly discrepant in terms 
of acceptable characteristics;, for both the rural and urban samples^ were 
eliminated from subsequent analysis. Items chosen for elimination are shown 
by asterisks in Appendix A. Measures in which one or more items were elimi- 
nated^ and the justification;, are as follows : 



* 

The item-total correlations of Appendix A were not corrected for spuri- 
ous overlap and should only be used as a general guide in item selection 
decisions, as they were in the present instance* 
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odb Kncv^led^ e ; Foui’ items in the male version vrere elimiiated; three 
because of difficulty levels (p values) of 1.00 in either rural or urban 
samples (i.e.. everyone ansv;ered the item correctly), and one because of 
negligible item-total te-st correlations. Of the foiu" items, three cover 
questions about tools used for the jobs pictured and even the one item re- 
maining in the ”tooi" category is at a p value of greater than .9t foi' the 
I'Ural and urban samples. Obviously the entire category should be considered 
for elinination in future development of the measiue. Five items uere elimi- 
nated from the female version of the Job ?Cnomledge test; four because of 
aifficulty values of 1.00 and another owing to a negligible item-total cor- 
relation- iliese five items are not found to be primarily in any one job 
knowledge category. 

Self-Esteem : Elimination of one item each, in the male and femaJe ver- 

sions^ was made on the basis of a lack of item-total test correlation; with 
no x^articular aspect of the item content to explain the inadequacy of these 
tv.'o items. 

Deferred Gratification : r'er the female version only^, there was a 

single item eliminated beca.usc of csserLtially zero item- total co^umla.- 

tions in the rural and urban s a': pies. 

Job Seeking and Holding Skills : A decision to drop a single item, and 

only in the fcrnale version, was based upon its p value of approxlmate-.d.* 

1.00 in both rural and urban samples. Tne it-m is one from the first Ij 
ite^'is of the booklet *.<hich is the Job Seeking Skills portion of the r.eas 'v-: . 
ant is a question about infoi'rnlion require i for ,Wi emplcvneent arT^l ic a' 1 :uj 
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Pi-actical Reasoning-File Cards : This is the one meas'*jire for which all 

items as a group could be considered unusable. The result occurred only 
for the female sample (urban and ruual) and it is apparent that the measure 
represents a task far too easy for ferjiale enroilEes. 'Ihe expected variance 
restrictions (for items at an extremely easy level of performance) are also 
apparent from the item data presented. Thus, this measure vas dropped in 
the subsequent analyses of results for the femle sample. 

Overall; relatively few items were elirrdnated and the measures vrould 
appeal*; on the basis of their patterns of item chai’acteristics; to be 
applicable to a rural as well as an urban sample of N-Y.C. enrollees. 

Urban -Rural Compariso n 

Prior to the major phase of the analysis , it v;as considered v.orthvrhile 
to compare resents for the rural and urban samples, based upon total mean 
scores for each of the enrollee measures. Fii’st; because as research find- 
ings; su?}i differences are of interest in their own right. But; the com- 
parison was also made because of its value for future develojimcnt and 
utilization of the measures; as well as to deternine whether or not the 
differences are minimal eno'ogh to justify combining the tvro subgroups into 
a single sarxple for data ami lysis p'.u'poses in the present study. VHiere 
significant differences do o cui" for pai’ticuiar measures; it is necessary 
to consider possible attenuation of correlation,: when combining the samples 
and the advisability of dividing rural and urban samples in future work 
with the measures. (iiopePaily. thougVj. there would be attempts to cross- 
validate any such differences v:ith a larger moral sa’'ple than w'as available 
here. ) 
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Table; 1 presents the means and standard deviations obtained on It 
ores from the 15 enrol lee test booklets. (llote that the two additional 
scorc-s result from presenting sepa.rate vali;es for the lob Seeking and Job 
Holding portions of the ''Job Seeking and Holding Skilis^^ booklet and a 
Difference Score based upon Vocational Aspirations and Vocational Plans.) 
Of the '30 tests of mean differences (It each for the irale and female 
group.^ ) there were only two that reached a degree of significance at the 
.01 confidence level for the miaie group and three for the feriaie group 
(based upon tv;o-tailed tests). The *05 level differences are shovm in 
Table 1 but considered at too lev; a confidence level to be v^ortli stressing 
with the present saT^iples. Thus^ urban males can be seen to score, signi- 
ficantly higher on the Map Heading sub test and rural males on the overall 
mean for Interest in Vocational Tasks items. (ihis latter difference 
indicates only a tendency of im.ral males to select any occupational tasks 
as more acceptable regardless of the par+.icular occupational clusters in 
v;hich they belong. ) Kural females achieved significantly higher mean Self- 
Esteem scores and gave higher ratings to their IMY.C. projects on the 
Pnrolloe Hating Scale. In suemar'y. the xjat.tern of item characterist ics 
along v:ith the relatf.'ely smiall n\m'.ber of significant mean differences 
that occur when ^ju^ban and rural groups are compared (as well as the pro- 
portionally small rural sample si.e) lead to reasonab.le justification for 
combining the two sa::iples. Tne remainder of the data analyses are there- 
fore based upon corbined rural and urban sar.-ples vritVi separate ar/^lys'. 
c^n 1 f o r m. ale ar; i f c rr ale s xT: g r c s . 
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Table 1 

iiTirollee Mc-asiu’es: Means and Standard 

Deviations for Roial and Urban Sainples 



Male p'^emale 



Rural Urban Rural Urban 



Measure 




(N= 29 ) 


(n 5 ^T 




(lil' 103 ) 


Job Seeking Skills 


Mean 

SD 


9-76 

5-50 


9 .95 

5.29 


10.00 

5-70 


11.71* 

2.99 


Job Holding Skills 


Mean 

SD 


29 . 99 
9.22 


27.86* 

5-55 


51.05 

1 . 9 ^ 


30.98 

2.35 


Enrollee Rating Scale 


Mean 

SD 


99-85 

11-79 


99-99 

12.82 


67.85 

6.90 


62.45** 

9.16 


Attitude t award Authority 


Mean 

SD 


4 i. 4 l 

9.02 


42.41 

8.5J+ 


42.89 

8.96 


41.51 

8.15 


Self-Esteem 


Mean 

SD 


57 -cr/ 
5.14 


57.76 

5.51 


59-75 

1.87 


57 - 99 ^''- 
2.99 


Deferred Gratification 


Mean 

SD 


95-05 

7-99 


91.76 

7.90 


55.65 

6.16 


95-59 

9-97 


Motivation for Vocational 
Achievement (l and II ) 


Mean 

SD 


4 0.00 
5.22 


59-18 

9. < 9 + 


40-85 

4.15 


40. '60 
4.68 


Job Knowledge 


Mean 

SD 


5.98 


22.86 
4 . 54 


21.17 

4.29 


22.23 

5.27 


Vocational 


Interest 


Mean 

SD 


79-17 

11.67 


72.50*=' 

11.39 


64.55 

8.95 


62 . 59 
11.78 




l!ap Reading 


Kean 

SD 


4.5J+ 

1.80 


0 . 02 * * 

2.40 


4.75 

1.87 


4.93 

2.19 


I’T'actical ^ 
Reasoning 


Zip Coding 


Mean 

SD 


9-58 

2.69 


6.48 

2.65 


7.41 

2.45 


7.16 

2.16 




File Card 
^ Sorting 


Mean 

SD 


6.48 

2.56 


7-56 

2.29 


6.50 

2.08 


8-95 

1.69 


Vocational Plans 
(Discrepancy) 


Mean 

SD 


9.62 

1.95 


6.24 

2.51 


7-25 

2.56 


6.64 

1.99 


Vocational Aspirations 
(Discrepancy) 


Mean 

SD 


6.10 

2.94 


6.96 

2.88 


7.80 

2.76 


7 • ‘>9 
2.12 


VA-VP 

(Dif fez'ence ) 


’■!ean 

SD 


7.10 

9.79 


6.05 

6 . 4 o 


7-90 

5.46 


9 - 51 
5.90 



^Difference between neans significant at . 0 !; level. 
^■^Difference between ricans significant at .01 level. 
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C* Reliability and Vali di ty o T the Kr.rollee Measui^es 



The internal consistency coefl icients , vmich serve as estimates of 
leliability foi' the moasu.res, are preserted in Table - for the male and 
female sajr.ples. Seve^'al approaches to comx^nting internal consistency are 
utilized. As indicated in the table,, they are ba^sed upon either the 
Speavir:?.n-Bro\'m foi'mula epp.lied to a split^-half correlation; Kuder- 
Richardson (hi) estinates Ter items scared on a right-vrong basis or, 
where there were very few itv'nis in a measure, the average iLem^-iutercorrela- 
tion was stepped-up (by the Spearman -Bre^wn formula) to the nmber of items 
in that msasui*c. 

Reliability estimates are, for the most pai't, reasonab3,y acceptable-- 
in some instajices substaritlal- -despite the I'elatively srall number of items 
comprising the various measur^^s. The obvious exception is the overall 
Deferred Gratification score with reliability estima.tes of o7 for males 
and .29 for females. However, it should be recalled that the measure v^as 
constructed on the basis of four sub scales of four items each. Uhen the 
reliabilities of these subscales are corpjuted (average item intercori-elatlon 
of the subscalc step-'ped-up four times) i"^ bc-co:ncs clear from hable h tiiat 
the attenuating effect or the total DG iwcasurc probabiy results frc:a the 
EUbscale designated as ’'Delay of Reinforcement" which has, essentially, 
zero reliability. ?h^ interit'un correlations for the subscalcs v:i]l be 
oxpmdned in detail in the datin' discussion of results fer the DG measure. 

Of note is that the re .liability csiiirates for Voc at ioi'.al TMans and 
Vocational Aspirations are a mx-asure of the inte7 correlations between 
i?airs of items at each of eight status levels. So that, :.n effect, ihest 
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Table 2 



Reliability Estimates for ^'-nrollee Measores 



M easur e 

Job Know'ledge ('JK)^ 

Voc€.tional Plans (VP)^ 

Vocational Aspirations (VA)^ 
Vocatioiial Intel est (Vl) 

(l) Clerical^ 

(0) Ser/ice^ 

(3) Technical^ 

(4 ) Outdoor"^ 

(t) Science’ 

{ 6 ) Business^ 

(7) Acstnctic^ 

At t it u de t Ov>:ar d '\‘j th' > r L ty ( >'\J\ ) ^ 

Self -Esteem (SE)^ 

Deferred Gi atif ication (DG)^ 

(-0 Freedom of Sp'ending^ 

(?) Hasty Aggression^ 

(^) Affiliation^ 

( '4 ) D e lay of Pe i n f ore- ^ rnc nt ^ 

Job Seeking Skills^ 

Job H'^lding Skills^ 

Motivation for Vocational Achievement 

Practical Reasoning 

(1) Map Reading (PKlO^ 

(2) Zip Coding (PRZ)^' 

(5) Kile Card Sorting 
tinrollee Rating Scale'* 





Males 


Feinales 




(H= 125 ) 


ani 33 ) 






d-tt) 




-75 


-72 




.81 


•67 




.78 


.67 




.62 


.80 




.69 


.65 




•39 


•71 




.65 


•78 




.60 


.69 




• 96 


.64 




.67 






.G, 


.87 




■50 


.60 




37 


.29 




•7'4 


.69 




.64 


•56 




-61 


.61 




.rt{ 


.06 




•69 


.66 




- 7 '^ 


,72 




.8.1 


.68 


(MVATT f 


.31 


64 




.60 


•W 




■76 


. 66 

d 




.7.1 






.96 


.90 



o 

ERIC 



^Split Half; stepped n]' by Spoanran'Bi ori formula. 

^Kuder -Richards or (Pi) estimates. 

Average Interitem r stepped up by Spear'man-Bi\r.>'n forrirUa. 
'^Dropped from use with feiriiiv:^ sairiplc . 



are the closest of all the coefficients to an alternate (or parallel) forn 
type of reliability estimate, they also represent a measure of the degree 
to w)iich consistency of status und^:rlies the vocational choices. This 
degree of consistency of choice can be seen in Table 2 to be rather similar 
for the plans and aspirations measures. The exceptionally high level of 
reliability for the Knrollee Rating Scale (r ~ .96 for males and .90 for 
females) is als'^ vorth noting and is indicative^ not only of extreme con- 
sistency in judgment^ but of considerable generalization or “halo” effect 
over the items that make up the scale. That is^ if the eri:*ollee's attitude 
tovai d the N-Y.C. project is favorable^ it tends to be a highly generalized 
attitude over all phases of the piogram. 

Tiie CO ‘relations that represent validity coefficients are presented in 
Table 5 for the male and female enrollee samples. These relationships 
betv;een the enrollee measur-es and the counselor and \>ork supervisor Lvitcr- 
ion ratings can be considered as no more than lov;-to-moderate in 
But is is also the pattern of thrsc correlations which is of interest for 
defining t'ne measures oi the battery that might be more predictive of 
longer-term enrollee ^^ocatienal or social adjustment. Ibey also provide 
clues as to the bases on which counselors and v;ork supcivisors evaluated 
the enrollee s. 

The pattern of validities is found to be fairly similar for counselors 
and work oupex visors^ although certain differences between ^hem arc worth 
noting --particularly in their evaluation cf male and I'ernale enrollee s. For 
both criterion raters, a clearly significant Sf t of relationships exists 
between their ratings of male onrollces and performance on the three practical 
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Table 5 



Vali^iity Coefficients : Correlation of Counselor and Work 

Supervisor Criterior. Ratings with linrollee Measures 



E nrollee Measure 

Job Krtov; ledge 
Att. toward Auth. 
Self-P^steem 
Def. Gratification 
Job Seeking Skills 
Job liolding Skills 
Mot. for Voc. Av'h. I 
Mot. for Voc. Ach. II 
Map Reading 
Zip Coding 
rile Card Softing 
Kn rod Ice Rating Scale 
VA (Discrepancy) 

VR (Discrepancy) 

VA-Vl (Discrepancy) 
Vocational Interest 
Clei i^?al 
Service 
Tecbmical 
Outdoor 
Science 
Business 
Aesthetic 



Male 



ounseloi‘ 


Woi'k Sup 


(N-llR) 


(n 5 i 02 ) 






. IT 


.22*- 




.01 




■17 






. 22 * 


■ n 


22 C** 


.21* 


. OT 


.07 


. 2 k ^ 


.28** 


. 20 ^- 


.in* 


22 y 


. di* ^ 


, 22 * 


221* 


- . 00 

riii 


-.0^ 


- • um 

-.r/f 


" » 

-.18 


• 17 


.16 


.0’* 


-.01 


-.n 


- . 08 


.oJ^ 


.01 


• !>;> 


.09 


• 01 


.Ih 


.01 


...1? 



Female 



Counselor 


\}otK Sup 


{N-129) 


{D- 128 ) 


.19^ 


.05 


.06 


.06 




. 21* 


.18 


^.05 




. .22* 


.26^ 


-17 


.21^ 


.15 


.10 


.15 


.06 


. 06 






.20* 


.15 


-' 7 A 

0 

1 


-,15 


-.11 


-.15 


.00 


-.Of 


.10 


.0:) 




- . 01 


.00 


-,01 


.c6 


.01 


.15 


.00 


-.oV 


-.09 


• Of 


..19 



sigrdficantly greater than zero at .00 Isvel. 
significantly gx'cater than zero at .01 level. 
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Reasoning S'abtosts; with larger correlations for the work supervisor ratings. 
Apparently, the male enrollees who have the ability to follow directions in 
carrying out tasks that also have some reading -skills component (as is char* 
actor is tic of the Practical Reasoning measures) arc seen as supci'ior. For 
fe?nalos, however, the abilities reflected in this type of ?noa.su 2 'o ai'c not 
significantly indicative of performance ratings by work supervisors, fnly 
the direction-follov7ing and numerical sort of task that makes up the /d]' 
Code subtest beai\s a lov; positive relation to counsc-lo7’ ratings of the 
females (r - -Fb). 

Consonant with these findings arc the Significant correlations bctv’o^ni 
ratew evaluations and the Job Socking Skills subtost (especially fin' r.aies) 
with an r of . jd for cconselor ratings and - lb tor v:ork su]:ervisor ratings. 
ftS'Se ly items, it should be recalled, consist juuniarily of employrment want- 
ads and sections of an cipplication blank. They arc also the most heavily 
v:oightcd of all the cni’oliee me^asures in a reading p. qui'^emont and consti- 
tute virtually the only mat-rria.l in t};e braltory wliich the enrollee T^ust 
read ct^tirely cl his own. 'Ujus, an individual who is superior in the skills 
r>'’quired for this s^'^t of Iteins tends, inufoi'mly, to be .Judged as a better 
enroll CO by courioe-Iors and vrork suC'Crvisors . i lic- tjn-: other measure that 
requires vrhat migl^t br considered sor/.o coripor.cnt of intellcctaa.1 capability 
(althov.gh not specifically verbal to any great extent) is th^: Job KriO’wlcdge 
neasutO. lliis also appears as positive .ly related to counselor and \cork 
supervisor ratir-gs for rriales but v;ith lesser validity for the females, 

Of tl’.e attitudi.nal types of measures, the best validity is gericrany 
found with thm counselor I'atirg criterion. Aex-ng mjales, the ;■ ignif ica*' t 
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cor relations j, in order of magnitude, are found for Deferred Gratificatj*.on 
(r - Motivation for Vocational Achievement i I (r = *2G) , Job Holding 

Skills iv - -22)^ and the Knrollce Rating Scale (r - 2?.2), pattern of 

significant relationships closely parai.lois these found for counselor rat- 
ings of the feriialc enrollees. V/ork supenAisor ratings of males present a 
pattei'H of validities reasonably similar to that of councolors (i.e.^ 
significant^ but low, correlations with OH, MVA:I and El^--although not 
quite significantly related uo DG ) • One aifforence vrorth noting is that 
Attitude tc^'ravd Authority reaches a significant level v;ith the vork super- 
visor criterion ratings of males (r - .22), It v;ouJ.d seem logical that 
vork supei'visors i>laco greater stress on this type of confoj'iaity among thi 
male enrollees. 

From Table j it can be ^een that v/itli one exception, th'~ fvur-item 
job clusters of the Vocational Interest Heasi,ire achievt^ n > particular 
validity for either males or females. The exception is that females v:ho 
tend to score high on the Service cluster also tend to be rated higher by 
counselors (r - . !?*;)• 

Ihe next section jjroyides a rriore coinprehensive viev; of the pattern 
of validity and its meaning for the d ''celo] merit £tnd use of the enrollee 
measures . 



D • Inter cor relations and the Factor latte rn 

Tne matrices of intercorroiations for ail of tlie evaluation measures 
in tlio enroJlee battery are shown in Tables and Tire 22 x 22 matrix 
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for the male and the 21 x 21 matrix* for the female enrollees were each 
factor analyzed in order to determine the underlying factors that make up 
the test scores. From the pattern of interrelationships and the cluster-^ 
ing of the various measures, a more precise statistical summary of the 
dimensions tapped hy each measure can he identified; as well as pertinent 
.clues to the most effective meas'jres utilized in this preliminary develop- 
ment phase. 

For both factor analyses ; seven factors were extracted hy a principal 
components solution and the factors were then rotated using a varimax 
rotational method (Kaiser^ I958).** The two criterion rating scales were 
added to the rotated matrix hy factor extension in order to provide an •' 
overview of the validity contribution of each dimension. 

In the male enrollee sample, which will he considered first; four of 
the factors had a sufficient number of measures with loadings at a reason- 
able enough level to allow for interpretation. Thr^ee of the four are 
rather clearly designed as Intellectual Skills (Factor l); General Voca - 
tional Interest (Factor II ) and Positive Attitudes (Factor III). The 
fourth factor^ which is defined hy relatively few of the measures in the 
battery;, is probably best conceived of as one indicating a I^egative Self 
Image (Factor IV). 



* 

The Practical Reasoning File Card subtest was eliminated for females 
as indicated in 'Section IV A* 

The seven factors aceo'onted for 69^ of the total variance in the 
matrix for the male sample and 66 fo of the total- for the female sample. 
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Total Score Correlation Matrix 
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Vocational Plans -10 



'i'ij*? yatU-rn r>l' i'or Factor as showi belov:, is >‘eadily defin- 

able froin th«.' av|.earancc oi‘ its highest loadings on the three Pr-actical 
Reasoning measures as ucD as on the Job Knowledge and Job Seeking Skills 
Measm*es. (Loadings of about or greater v:ere considered For interpre- 
tation thrvjujhout. ) 

Factor I 

Intellectual Skills 



Measure 


Factor Loading 


Practical Reasoning 




File Cards 




Map Reading 


.8 j 


Zip Code 


.80 


Job Seeking Skills 


• 79 


Job Kji owl edge 


. Cm 


Motivation for Vocational Ajhievemeat (l) 


•99 


VI: Outdoor Cluster 


-.99 


Job ilolding Skills 


09 


Deferred Gratification 


.19 


Attitudes tov:avd Autiiority 


09 


Work Supei’visor Ratirig 


•99 


Coiuiselor Ratings 


.ii8 



A niurjber of t}.e attitudlnal measures also make a eon'O I'uution (although 
a relatively minor one) to this factor. 'Hieir loadings ouvioi:s]y confirm 
vhiat appears in the intcrcorielation matrix, v;here tljosc altitudinal 
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TT-casuTes aro oeen 'cc havf lo>7 t-o ijii d6rat;6 corr6j.ations v/i^th ishe. iiitellccl j 
slfi. lls type of measures. 

The one loading that; appears for the Vocational Interest jot cluster : 
is on the "Outdoor” category. Its negative leading seems logical as an 
indication that niale enroliees who score hig: on the Intellectual Skills 
factor tend to score low on (i-e.^ reject) the sort of job tasks that miok 
up the Outdoor clu 3 ter--the tasks pictured ocirg unquestionably lower levels 
or menial. 

Probably of major interest in this patte^’n of leadings is that the 
Criterion rating scales for counselors ar>d work supervisors also achieve 
loadings cf interpretable magnitude on the factor. As will be apparent 
frem the remainder of the analysis^ this is the only factor for males to 
which these criteria make a significant contribution. It is clear that 
these raters tend to oe more favorable in their evaluation of male enrcllees 
who score higher on the sort of measures that define the Intellectual Skills 
factor and, in addition^ that it is the work suix?rvisor who seems to place 
greater emphasis on this area of capability. 

Fcr ?vcto-‘ II (General V ocational Interest ) the pattern Ci’ loadings is 
an ob\'ious reflection of the intercorrelations between the job <"Vaster3 of 
the ’'inteiest in Vocational Tasks” measure. Favorable choices or acceptance 
of tasks in job clusters tend to generalise over virtually all of the VI 
clusters. As might be expected, the Outdoor cluster constitutes the excep- 
tion and is not found to be loaded on this factor. 
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Factor II 

Gej:''"ral Vocational Interest 



Measure 


Factor Loading 


Buf^'iness 


.78 


Science 


.76 


Aesthetic 




Service 


•75 


Clerical 


-70 


Technical 


.67 


'VP (Difference ) 


01 


‘ollee Bating Scale 


-24 



An Tuiusual addition to the ]oadings is the VA-VP Difference score 
which indicates that the enrollee with generally positive occupational task 
interests is one who has a larger difference between his vocational aspira- 
tions and plans. (For these data, as wild be shown in later discussion, it 
also means, specifically, that the tendency is for such a difference to be 
in the direction of higher a,spirations . ) Individuals who score highex' on 
the Enrollee Bating Scale also tend to be those who have a pattern of 
broader vocational interests. 

Factor III has its lead loadings on the unmistakably attitudinal nea- 
siu'es of the battery and represents a dimension of Positive Attitudes - 
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Factor ITI 
Po s it ive At t itude s 



-lob Holding Skills 
Self -Esteem 
Enrollee Rating Scale 
Attitude to\'/arcL Authority 
Motivation fo?' Vocational A.cnievement (l) 

Job Hiovnledge 
Job Seeking Skills 
VA-VP (Difference) 

Two of the measures that are concerned primarily with intel3e^tuax 
skills (job Seeking Skii^.s and Job Knowledge) have a minor role in the 
factor- -again, indicating the modest overlap between attitudes and intel- 
lectual skills for this group. Hie VA-Vp Difference score also makes a 
minor r ontribution to th^- factor, indicating that a high score on tne 
attitudinai dimension is ac corr.pdrdcd by a tendency to shevj a Birger dif- 
ference between vocational aspiratJons and plans. It should also be 
noted that there are no leadings of reasunatle Ti;agnitud.. found on this 
factor I'or counselor and work 3upc?rvisor ratings. ihose individual 
attitudinai measures that do correlate significantly with the criterion 
ratings apparently do so largely because of the intellectual component 
that they also possess. 

The fourth factor found for male cnrollees is d . fined by only five ^'f 
the m-.asures but sce.ms rea.sonably intcrpretable. 



o 




Factor Lo aa ing 

^70 

•TO 

.66 

• 5^ 

.47 

•39 

.p8 

• 28 
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Factor IV 

Negative Self Image 



Measure 


Factor Loading 


VI : Outdoor Cluster 


• 75 


VA-Vl' (Difference) 


-•55 


Vi; Technical Cluster 


• 55 


Aotitude toi-rard Authority 


-50 


Self-Esteem 


-.28 



The description is essentially one of an irdividual vho is strongly 
favorable in his choice of the Icrv -level, menial type nf tasks that define 
the Outdoor Cluster of the VI measure and who sherws little difference be- 
tween his vocational aspirations and plans. He also tends ^ to a minor 
degree,, to favor the tasks of the Technical job cluster; to have a higher 
score on the Attitude t,.\rard Ai^.thority Pleasure and a Icns^er wSel.f-Esteem 
score# In essence,, this seems to define the individual with a somewhat 
diminished self -concept or Negative Self « 

The factor pattern changes somewhat for the femle enrrliee sample, 
with five factors having a sufficient number of loadings of sufficient 
magnitude to warrant their interpretatj on* 

’^actor I is readily designated as the Intel lect ual Ski ll s Factor and 
is certainly comparable to the one found for the males. 
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Factor I 

Intellectual Srcills 



Measur e Factor Loading 

Job SeeMng Skills *80 

PR Zip Code .79 

Job liiovledge .76 

PR Map Reading .67 

Moti\aticn for Vocational Achievement (l) .55 

Job Holding Ski 13c .29 

VI Service .27 

VI Outdoor -*26 



Here again_, there is the minor contribution o"^ measures that are 
primai'ily attitudinal in content and also the negative loading for the 
.IcT/f- level basks of the VI Outdoor clustei’ on this dimension that reflects 
a form of intellectual skil.l* A unique addition to tne factor for females 
is the VI Service cluster. Counselor and vork supervisor ratings make no 
contribution to this factor worth considering. 

Factor IT corresponds ve-^y closely to the Ge neral Vocational Interest 
dimension found for the male sample and is designated in the game way. 



o 
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Factor II 

General Vocational Interest 



Measure 
VI Technical 
Aesthetic 
Sei'vice 
Outdoor 
Business 
Scienvje 



Factor Loading 

.B3 

•71 

•59 

• 51 

• 50 



Ihis generally favorable attitude toward job tasks extends over all of 
the occupational clusters, except for clerical, which seems to occupy a 
unique role for feiiiales. The girl who scores high on the clerical cluster 
does not tend to favor all of the job tasks indiscriminately. In fact, 
the relatively high positive loading on the Outdoor {menial tasks) cluster 
would appear to characterize this factor as one of low-levei, indiscriminate 
task interest. Support for this contention is found to some extent in the 
presence of a third factor which displays its positive loadings on the VI 
clusters uf Clerica.'., Business and Science and is negatively loaded on the 
Outdoor cluster. The facoor is designated as H igli Level Occupational Inter - 
est and Motivatio n (Factor III). 
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Factor III 



High Level Occupational Interest and Motivation 



Measure F actor Loading 

VI Clerical .69 

Vocational Plans (Discrepancy) .65 

VI Business . 60 

Science .40 

Outdoor - . 54 

Motivation for Vocational Achievement (l) .30 



Tne female enrollee defined here is one who scores high in interest 
on the vocational tasks in clerical^ husiness and science areas ^ is unfa- 
vorable toward the outdoor tasks ar.d scores high on Motivation for 
Vocational Achievement. However, the high positive loading on the Voca- 
tional Plans discrepancy scor^ indicates a somewhat puzzling tendency 
tovrard inconsistency in status choice- 

A factor of particular importance for the foraale sairiple is one that 
is eosil^^ interpre table as Pcsiviye Attitudes (Factor IV ). The added 
importance stems froiii its being the only factor that begins to .shcrw a 
loading of any reasonable level on a criterion rating scale- 
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Factor IV 
Positive Attitudes 



Measure Factor i^oading 

Enrollee Fating Scale .75 

Self-Esteeiri 

Jot Holding Skills .44 

Motivation for Vocational Achievement (l) .44 

VA-VP (Difference) -.27 

Motivation for Vocational Achievement (ll) .26 

Work Supervisor Rating .27 



ITiis highly positive attitude reward the N.Y.Ct program (Enrollee 
Rating Scale) and toward oneself (Self-Esteem), along with favorable 
attitude toward the ^ob situation (job Holding Skills) and toward employ- 
ment in general (Motm ation for Vocational Ac?iievement seems to define 
the female enrollee who is more likely to be seen favorably by the work 
supervisor. This is in notabJe contrast to the factor pattern for males, 
for whom favorable evaluations tend tc be related to measures that define 
the Intellectual Skills factor with no such validity appearing for the 
attitudinal factor. (The female sample also seems tc produce a ''cleaner’^ 
distinction between the attitudinal and intellectual components--so that 
attitudes do not appear as highly interrelated with intellectual skill as 
for males . ) 

The fifth and final factor interpreted here is best considered as one 
that defines S ocial Conformity (Factor V)* 
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Factor V 



Social Conformity 



Measure 



Facto r Loadin g 
•78 



Attitude to’.^ara Authority 



Deferred Gr at i f i c f .t i on 



.75 



Jot Holding Skill 






Motivation for Vocational Axhievement (T) 



‘59 



Although significantiy loaded on only four measures^ its interpreta- 
tion seems clear. More favojable attitudes tov;ard authority and toward 
the social values that define gratification deferment^ constitute the 
primary bases for defining the factor. Logically supplementing these are 
the positive tendencies tovic±rd. social conformity that would ocem to mark 
higher scores on the. Job Holdir g Skills and Mot taxation for Vocational 
A c h i men t me su r ■ e s . 

' The Cr iter i on Scale s 

Since these tv;o i^atirg measures serve as th- only independent stand- 
ards criteria) of -rr"o.[.U6 capability, it is desirable to consider 

their scale characteristics .in some detail as >;ell as certain aspects of 
their value as criterion measures. 

Ihe .H items that make up each scale (Appendix C) can be seen to con- 
sist of l(j fairly specific forms of enrollee behavior and one filial item 
representing a surrimary c-vaiaation of enrollee capability. Tne means, 
standard deviations and item- total score correlations are presented in 
Tables 6 and 7* 

o 
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Table 6 

Counselor Rating Scale: Item Characteristics 



Male 







(KS112 


Item 


Mean 


S.D. 


1} 


3-91 


.88 


2) 


3-39 


1.23 


5) 


4.14 


1.12 




3.97 


.99 


5) 


5.19 


1.14 


6) 


5.4 o 


1.09 


7) 


5.54 


1.01 


8) 


3.82 


1.15 


9) 


5.65 


1.25 


10) 


3.45 


1.37 


u) 


5.5!^ 


1.14 







Female 

Item- Item- 



Total r 


Mean 




Total 


.52 


5.92 


.85 


.78 


.54 


4.17 


•99 


.70 


.63 


4.18 


' .95 




.70 


3.76 


1.15 


-77 


.68 


3.61 


1.21 


.85 


.27 


4.37 


■ 75 


.67 


.62 


3.76 


1.21 


.45 


.80 


3.6J* 


1.14 


.81 


.46 


4.57 


.98 


.57 


.70 


3.97 


1.13 


.66 


.85 


3.93 


1.10 


.85 
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Table 7 

Work Supervisor Rating Scale; Item Characteristics 







Male 

(1^102) 






Female 

~(nS128) 




Item 


Mean 


S.D. 


Item- 
Total r 


Mean 


S.D. 


Item- 
Total r 


1) 


If. 22 


.90 


.59 


4.24 


.77 


.84 


2) 


5.50 


1.23 


.50 


4.13 


1.16 


.57 


5 ) 


4.26 


1.05 


.a 


4.36 


.79 


.46 




4.21 


.94 


• 71 


4.16 


.80 


.78 


5 ) 


5.45 


1.28 


.80 


5.78 


1.20 


.76 


6 ) 


5.75 


1.06 


.55 


4.55 


.68 


.67 


7 ) 


5.72 


1.12 


.47 


5.96 


1.25 


.55 


8 ) 


4.07 


1.06 


.81 


3.81 


1.04 


.79 


9 ) 


5.97 


1.14 


.68 


4.60 


.89 


.45 


10 ) 


5 . 8 y 


1.28 


.63 


4.26 


.96 


.71 


11 ) 


5.94 


1.16 


.86 


4.17 


.96 


.87 



i 
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Item-total score correlations are substantial an:l reflect, to some 



extent^ the exti’cmely high split-half reliability cstir.ates found. Ihese 
estimates (ste^jped up by the Spearj^lan-Bra■^^n foi’mula) are for counselor 
ratings and .9^ snpei^visor ratings. P> cr though .uch split-half relia- 
bilities tend to overestimate somewhat (Ttiorndike^ J 99 l).* they nevertheless 
indicate a gross "halo" or generalization effect on tie p^ot of the raters. 
Thus^ a favorable attitude tward an enrol lee tends t ■ result in a favorable 
rating no mtter what the specific behavior being evaluated, llie ccnciusion 
is further supported by the item-total scale intercorr ^lations for the sum- 
iTiary item in the scale (item jfll). Correlations of . 8 ^ for both male and 
female samples on counselor ratings and .86 and .87 for these samples on 
the v;ork supervisor scales indicate that the one sumiiria/ry item is almost as 
effective a measiire as the I'emaining 10 items combined. Not only is the]"e 
gross generalization in the evaluations hut the fairly extieme leniency 
shorn by the means of Tables 6 and 7 also point to a difficulty in obtain- 
ing discriminating judgments for the va^'ious characteristics rated-- 
especially on the part of work supewisors . 

Counselor mean rating scoies, overall, are 9.6 for males and 5.9 for 
females on the 9-l>oint scaled itemis. VJoik SLiporviso.v isean "catirg- are 5*9 
for ifLales end U.l for females. This rating leniency also tends to lestri.ct 
variability for th^ scaD.o itemr* One coulij assune that the w^ork supervisors 
simply see enrollees a.s distinctly superior in all characteristics covei'cd 
by the items* But this seems gratuitous and it is more likely that they aio 
reluctant to rate enrol lees imfa/orably on a fojTnal measure. More careful 
techniques of administering the scale could prove effective - -such as in- 
stnictions which stress that rntors try to assign a v:ider range of v'oight.s 
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for the items; plus specific assnxances that the results vouDd not affe- t 
an enrolJ.ee ^s status in the prugraia. Unfortunauely^ there vjas total depen- 
dence upon N.Y.C. project counselors leaving the rating scales with the 
work supeivisors and having them returned when completed^ so that only the 
printed instructions on eacli scale were a.vailahle to the work supervisoi 
and these are minimal (Appendix C). 

Agreement hetween counselor and work supervisor ratings of enrollees 
(interrater reliability) is found to be relativel^^ modest with a correla- 
tion betw^een the total scale scores of . for the males and .>6 for the 
female sample* Ihis rather mediocre consensus regarding an enrollee^s 
degree of “successful" or "unsuccessful" functioning represents an arguirier.t 
not only for improving the mating instruments (and perhaps thp judgmental 
ability of at least one of the groups of raters) but validation of coun- 
selor and wm’k supervisor judgment against longer range (follow-up) per- 
formance measures for N.Y.C. enrollees* 

F . biscussion of Specific Knrollee Measures 

On tlie basis of the results already discussed^ and additional data 
analyses,, it is of val\ie to place these findings in gcncrEl perspective as 
they apply to each of the enrollee measures* Foi ease of discussion^ the 
results are considered first for those measures found to define the intel- 
lectual skills dimension, fcllcr^ed by those most readily classified as 
altitudinal. 

Job Knowledge : 'ihe me^isure can be considered one of the most effec- 

tive of the battery in terms of its reliability (in the .70's) and its 
correlations with the otlm'r enrollee measures^ as weJl as with the criterion 




. 80 



- 76 - 



ratings. Its relationships cut across both the other intellectual skills 
types of measures and the attitudinal ones. The levels of these correla- 
tions are relatively high^ particularly with the Job Seeking Skills meas^are 
and the .'tactical Reasoning measures. For the male enrollee sample it 
tends to show more significant correlations with the attitudinal measures 
(than it does for the females) and is also more highly correlated with the 
criterion ratings given by counselors and work supervisors. 

An additional finding that may be of value regarding enrollee knowl- 
edge of job requirements can be seen in the scores achieved on the various 
categories that constituted the scale--i.e.j Place (where the job is per- 
formed), Natixi’e of the Work, Working Hours, Education Required and Start- 
ing Salary (for full-time employment). The "Tools'' category is not 
considered here because of almost no errors having been commicted in the 
responses for that category. 

The proportion of correct responses in each of the five Job Knowledge 
categories is as f oilers s : 



Category 


Males 


Female s 


Place 


.90 


•95 


Nature of Work 


.85 


.88 


Hours 


.76 


.78 


Education 






Salary 


.51 


.60 



These proportions appear to speak for the),iselves and generally indi- 
cate that the weakest areas of job kno-A^ledge occm" in the categories of 
Houi's, Education and S'^.la.ry requirements. Hov,^ever, such a generalization 
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should be tempered, by notirig that^ for the fairly common occupations 
depicted in the itams of this rsk-asure , it is not easy to construct reason- 
able distractors for items that ask ‘'Where would you work on tiiis job?-' 

On the other hand, item difficulty level is nruch more readily controlled 
when the distractors represent salary range « In any events the possibil- 
ity of eliminating the ’’PlisC:/' category of items would be open for con- 
sideration on the basis of these data. 

Practical Rea s oning Subte sts: The three jvasures that have been 

intended to assess practica’ reasoning provide a somewhat di'^fex’ent 
picture of I’esults for the rials and female groups, for males the three 
measures (Map Reading, Zip Code and File Card Sorting) form a highly inter- 
related cluster and would apT:ear to measure skills with considerable over- 
lap- Their level of reliability can be considered fairly good and validity 
-was sho^vTTi to be at a con^ iste ntly significant level ’'^or both counselor and 
\rork supei'visor criterion j^atings - -with ajjpreciably better validity for 
their relationships with vsork supe". /isor ratings. 

As m>.asure5 that show the highest loadirigs on ore of the major factoids 
of the battt ? y they ser^e to aefin-; a cie^ir intellectual skills factor 
althm./gh they are also found to be moderate.ly related to certain of the 
vocationally-oriented attitudinai measures (i«e.j Job Holding Jlkills and 
one of the Mot fat ion for Vc'cationai Achic'/em^rnt suite sts). There is rea- 
sonable evidence for a favorab.ic ro-^catial role, by these lYactical Reason- 
ing m<=asui’t:s, as evaluative iti,;traTi,-; nt s arl as pikdictoi^s of rale errollf-c 
pcrfor:.:ance in a job bmtt-ing. 

AjTiOng female enmollees, tne ITactical reasoning rrkasure-s did not 
prove as successful, first, Iv cause of the complete inadequacy cf the File 
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Card Sorting subtest as a measure (far too easy a task) and^ in addition^ 
because the two remaining PR sub tests (Map Reading, Zip Coding) show a less 
dominaiit effect in defining an inte.llectual skills factor. They are also 
not as consistently related to the criterion ratings, as in the male sample, 
and they are not as highly correlated with one another (r = .41.). Neverthe- 
less^ the item characteristics of both measures are suitable and their 
pattern of significant intercorreIation=; with the other measures is exten- 
sive enough to warrant retaining both of them in the bittery for future use 
and analyses with female enrollee samples. The more useful measure of the 
two for females would, generally seem to be the Zip Code subtect. 

Job Seeking Skills : This ma.jor contributoi' to the intellectual ebil- 
ity dimension of the battery appears to achieve its status primarily because 
of the rec^uirement for verbal reeding skills that it imposes on the 2‘espon- 
dent. At 1 he same time, hon;ever. the Job Seeking measure possesses one of 
the most consistent patterns of positive relationships with the attitudinal 
measures. Specifically, this occurs with the Attitude towai'd Authority 
measure for itiales with the tvro attitude measurers having the highest voca- 
tionally oriented content'-i.e., Job Ho' ling Skills and Motivation for 
Vocational Achievement (r’s in the .^O's) for both and ferules. The 

reliability of the measure appears more than adequate for future use, with 
its r^^ of .65 for males and .66 for females. Of particular note, herwever, 
is that this Job Seeking Skills subtest constitutes the teat overall single 
predictor in the battery of both the counselor and work supervisor criter- 
ion ratings. 

Attitude terward Authority ; A favorable pattern of item ch=>,racteris- 
tics (means, va^uances, item-total test r’s) and substantial reliability 
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estimates^ for both male and female samples^ chai^acterir^e this attitudinal 
measure. For the male enroLLee sample^ it shws a number of significant 
relationships to other attitude dimensions thro'oghout the battery , as well 
as the highest single cori'elation of any attitude measure with the work 
supeinrisor ratings (r “ .2^?). For females ; although the measure does not 
display any particular validity, it does constitute the major influence on 
the fairly distinctxve dimension of Social Conformity . In general, it 
possesses sufficient consistency, both internally and in the pattern of 
interrelationships with the other measures, to warrant its retention-- 

■ especially for the male enrol lee group where it appears to have promise 
for predicting later vocational adjustment. 

Self -Est eem: One of the featvres of this measu2*e is the relatively 

’’pure" attitudinal aspect of its contribution to the battery. Airiong the 
males it can be seen to correlate significantly with only attitudinal 
scales (,DG, -JH, MVA : I, ERS and the VA-VP Difference score). Since mea- 
sui’es 07J the attitude dijnension were sha/S; to \ assess little validity for 
males,, this is found tr> be especially true for the Self-Esteem measure 
which is e'xcluiively attitudinal in its factor contribution. As might be 
expected from the factor patter*, for feuiales, and from the modest validity 
achieved for the Fositive A tti tude dimension (on v'lich Self-Esteem had the 
lar*gost factor loading), it dees make a strong contribution to what little 
validity the battery achieves lor the fer, \le Si^mple. V/ithin the battery 
its intercorrelations are highest for the most vocationally oriented in 
content of the attitude measures- -i.e . , Job Holding Skills and I’otivatiop 
for Vocational Achievement ; I. Self-Esteem, for the feiriale sa-mple, a.lno 
shews relatively rare significant intercoi’relations with job clusters of 

o 
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the ^^Interest in Vocational Tasks’' measure (r of ,23 with the Clerical 
cluster and .21 with the Business cluster)* In general, the Self-Esteem 
scale possesses a respectable pattern of item characteristics and a suit- 
able level of reliability^ although it might be expected to have more 
potential, value vdth a fcma.le enrollee sample. 

EnroULee Rating S cale: Similar in a n^jmber of. its relationships to 

the Self-Esteem meas'ore, the Enrollee Rating Scale was found to ser'vre 
almost- exclusively as an altitudinal measure. The major difference is 
that it results in a greater mmiber of significant correlations with the 
criterion ratings, i^mong the measures used with male enrollees^ ERS vras 
found to have a considerable number of significant and logical inter cor- 
relations^ but is particularly weak in this respect for the female sample. 
Of relevance in the pattern of intercorrelations for males is that the 
relationships are strongest with the, vocational - c ontent type of measures-- 
i.e. , Job Knowledge (r - Job Ifolding Skills (r - ) and Motivation 

for Vocational Ac'aievement : 1 (r = .42), 

To some extent, the faiiui'c of the EnrollTe Rating Scale to cojuelate 
vrith numerous other measures used for ti^e female saurpl^ is likely to have 
^'esuited from the unusually high item mean values (i.e., rater ''leniency'') 
and consequent restriction in variance that resulted. Even with this 
limitation .♦ the scale has a valuable rune t ion in the batte.ry and would 
prove even more useful for ferrales if the scale format and lnstr;.c tions 
are revised to elicit more iis: U'ininating (i.e. , cidtical) levels of 
response. 

D eferred Gratification ; Despite the built-in multidimensionality of 
this measure (i.e.. four separate subscales), the DG tota.l score seems to 

o 
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play a logical and consistent I'ole in its relation to the other iricasures 
of the battex*y. For both male, and female c-rrrollees^ it is one of the 
attitadinal measures that displays a minor intellectual-skills component 
(as seen in its factor loadings^) and aumong females it also provides a 
major contribut:on to the attitadinal dimension of S ocial Conformity * Ihe 
level of validity for DG is found to be significant only for males and 
primarily in relation to the co^unselor criterion ratings. 

AJ though its reliability estimates v^ere fo'ond to be the lowest in the 
battery^ this poor reliability for the total DG scores was sho>,m to be 
attributable for the most part, to a lack of any reasonable level of in- 
ternal consistency in one of the subscales. Tliis attenuating effect is 
the likely^ result of having incorporated the group of items that make up 
the subscale defined as “Delay of .leinforcement” (item numbers 4^ 1.1^ l4 

and l6 in Appendix k for the DG measure )v these individual items also 

less tnan outstanding item-total test correlations in the DG measure. 
The item inteccoiu-eiations for scores on each of the four sub scales 
that constitute the DG cv'^acure are shovo'i for the i;iaie enrollee sa:nple. 

(Tnc pattern for s is essentiaLLy the cajne*) I'he intent’ is to pro 

vide an illustration of the- value, and coherence of each of the sub sc ales 
and to demonstrate the probable attenuating effect of these itenis on cor- 

I'elpAions computed with the total scoies. Table 6 presents tViese item 

lntercorrelHtio:‘*3 for the four subscales v;bich are designated as “Affilia- 
tion," “.Freedom of Sp^ending," “Hasty /ggression" and “Delay of Feinforcemen 
Cluster II designated as “Freedom of Spending" is the most coherent of 
the fora’ v?ith the highest item interccrrelations . Obviously the “Delay of 
Reinfoicement" cluster (IV) does not represent a sub scale at aLl. 

o 
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Table 8 

Item Intercorrelations for Sub sc ales of the Pef erred 
Gratification Measure (Male Enrollee Sample) 

N = 123 





I. Affiliation 




II- 


Freedom 


of 


Spending 




Item ^ 






Item 


# 






1 9 


13 


15 




2 3 




7 


8 


1 


.19 


.22 


.25 


2 


.33 




.41 


.22 


9 




.43 


.24 


3 






• 55 


.41 


15 






■ 58 


7 












III- }lasty 


Aggression 




IV. 


Delay of 


Reinforcement 




Item # 






I tern 








5 6 


1C 


12 




4 11 




14 


l6 




.14 


.26 


...2 


4 


-13 




-.05 


.21 


6 




.41 


,22 


11 






-.01 


-.,16 


10 






,4? 


l4 








.02 



The overall performance of the DG measure in the battery of errollee 
measures (despite the probable weaknesses iaiposed by one group of items) 
indicates that this attitudinal s>'ale ir worth further development effort- 
This should be based upon incorporating the three best subscales with some 
Judicious modification^ or item reconstruction in order to improve their 
coherence where possible. 

Job Holding Skills : This second portion of the O'ob Seeding anc. Hold- 

ing Skills test booklet is most apt.ly considered an attitudinal tyx'€ of 

o 
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rncaswe involving knowledge of proper behavior on a rather than one 

requiring intell.ectual or factual knowledge. This conclusion is verified 
by the relatively high i*elationships between -J}{ and the other attitudinal 
measures of the battery. Bu'*' it is also seen to have modest correlations 
with several of the intellectual skills measures and a corresponding con- 
tribution to the intellectual ability dimension. Jr. addition^ J}] shOT/cd 
modest validity coefficients for the counselor and v^ork supervisor criter- 
ion ratings as well as reasonably acceptable estimtes of reliability. 

Come possible difficulty is worth noting in the item :;haracteristics for 
the fetriale sturplt, v;hercin item mean scores tend to be overly large (i.e.. 
females tend zo give the most 'tiesirable'' or socially acceptable response 
choices on almost all of the II items). Thus^ extension of the scaled 
response choices to four or five alternatives^ as opposed to the present 
three, should be considered. 

Motivation for Vocational Achievement : The separate analyses of the 

11 items of MVA;I anvd the seven items (with their differing formal) that 
make up clearly indicate that MVA;T> as an item-format approach, 

results in a far superior measure. On the basis of the pattern of inter- 
relationships within the battery for iT^A: IT, and its correlations vfith the 
criterion ratings, the meas'ja'c appears to have little promise in any future 
development or application efforts. MVA:!, on the other hand, is a measu70 
with a strong influence thro^jighout the battery especially for females and, 
in general; has the best validity cjofficients of eny of the attitudinal 
measures, ilctabie are the striking correlations of with the Job 

i;olding CkiJ.ls measure (r - tor the males and for the female carripl^: ) 
as well as substantial dr give of positive relat ionships (among fcmalos) 
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with the Clerical aid Service occapationa], clusters of the "Interest in 
Vocational Tasks" measure (r = .4b and *52 respectively). Item character- 
istics and reliability estimates further reinforce the suitability of this 
measure as an instrument for use with N.Y.C. Out-of -School enrollees. 

Interest in Vocational Tasks : The seven occupational clusters of 

four items each that make up this measure formed, essentially, their own 
lanique factor of vocational interest and shov:ed relativel;>" fev: significant 
intercorrelaticns with other measures of the battery, The pattein of 
factor loadings and those few significant intercorrelations with other 
measures that were found are of value in deciding about the further 
development of any particular vocational cluster. Tor example, the some- 
what unique aspects of the Oi^tdoor cluster- -that have alieady been con- 
sidered- -might lead to dropping of this group of items from the VI measure 
since it seemis too heavili influenced by the perceived status of the jobs 
(i.e., low level, menial tasks) rather than by task interest per se . 
Clerical, Business and Service task clusters were found to be of particular 
importance for the female interest patterns; while Technical, Science and 
Business clusters among rriales provided almost all of the significant inter- 
correlations v/ith the o. - measures of the battery. Validity for any of 
the VI clusters is almost nil with one exception--the significant correla- 
tion of the Ser-/ice cluster for females -with counselor ratings (r = .25)- 
The particular" ly small number of items that comprise each occupational 
cluster, along with the imperfections in the criterion scales, -would not 
lead one to expect a patten of extensive intercori'elations betvreon these 
two sets of measures. 




Sd 






With regard to f"w:it“are application of the VI pictorial-verbal meastire 
a question arises of the smtabill.ity of the occupational, clusters used heir’e 
for SL’.oring purposes* The problem is one of whether the grouping of items 
based upon the all -verbal Califoraia Occupational Preference Survey (Khappj, 
Grants & Demos.; I966) (as derived from a primar*ily middle-class sample of 
students and factor analysis of the data) is optimum for a group of dis- 
advantaged adolescents- Tne question is one of whether the perception of 

f 

occupational tasks in a disadvantaged adolescent;, out-of-school group; 
differs in ways that might suggest different clusters for scoring purposes. 
This is especially relevant where pictorial information is added to the 
verbal content of a measuive and may lend different meanings to any occupa- 
tional tasks presented. Differences may also be engendered by the extent 
to which a respondent depends u.pon a *^^erbal or pictorial information mode 
when both are pra\?lded. Accordingl.y.. the 28-item male and female VI scales 
were factor analyi^ed, using a principal components solution. Eight factors 
were extracted from each 28 x 28 item matrix and subjected .to a varimax 
rotation. For each solution five factors are interpretable. GeneraLly, 
only four or five loadings reached magnitudes of a reasonable level on each 
factor - and the loadings of these items on the factors are shorwn in Table 9 
for males and. Table 10 foT females. 

Factor I for males is best termed a High Level Technical-Business 
cluster with a strong focus on the I,evel of the skil.ls required rather than 
the particulair a,pplieation of the task a highly skilled scientific 

task tends to be seen in the same way as a highly skilled business or 
technical task). 
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'^‘able )) 

Factors and Factor Loadings for Items of the VI Measure (Males, N= 123 ) 



Item Loading 

Factor I — High Level Technical and business 

Give advice on how to design a new airplane. .67 

Write up reports for a company on how much money it's making. *69 

Read dials and adjust switches in a research lahoiatory. 7 ^ 

Take charge of the building of a large bridge. .5/ 

Factor II--Low-Level (Outdoor) 

Feed cows, tfke care of them and milk them. 

Plant larus, shrubs and bushes and take care of them. . 7 i 

Do truck garden farming. * 7 ^ 



Do things for sick people in hospitals to make them co’.rforta'ble . »hk 



Factor III — Office-Clerical 

Run an office machine that handles all kinds of paperwork. 
Sterilize instruments used in a laboratory* 

Be responsible for taking cs.re of office files. 

Sort mail in a post office. 

Give advice on how to design a new ?\rplane. 

Factor IV--Technical Ski.lls 
Inspect r:>a''hinery. 

Cut and joir metal vuth a welding torch. 

Fix wires on telephone poles. 

Put plaster on walls and ceiJdngs. 

Take charge of the building of a large bi'idge. 

Factor V--Personal Service 

Help poor families that have problems. 

Be a department manager in a store. 

Sell automobiles to people. 

Help sick people in hospitals keep busy by shoving them 
how to make things* 

Help people rruake p>lans to go on trips. 



.67 

.9- 



. -a 

• 4 B 



.k 2 



•70 

-78 

.49 

.1+2 

•78 



.1+6 



..’+5 

• >5 
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Table 10 

Factors and Factor Loadings for Items of tho VI i’''oasui'o (Ferriales, K-153) 



Item 

Factor I--C]orical 

Be responsible for taking care of office files. 

Rim an office duplicator or copy macbine. 

Keep records of stock and supplies. 

Vh’ite up financial reports for a compuiy. 

dandle money thr t people put in and take out of the baTik. 

Factor II--I/>W“Lev'eJ. (Outdoor) 

Plant ajid care for shrubs and f lovers. 

Raise and take cai'e of bees. 

Feed cots, take care of them and ^mllk them. 

Cut and Join metal with a veldifig torch. 

Run a pressing machine. 

Factor III- -I-ow -Level Technical Skills 
Bake bread in large oven. 

Sell cosmetics to women at their homes. 

Run a pressing Tiachine. 

Sot up ea'tificial flowers in a store display. 

Fix Jewed-ry and watches. 

Factor IV- -High -Level Technical 

Do illustrations for a magazine. 

Collect iriforr abioji about weather conditions. 

So 11 i- - '-os c s to I c ople . 

Cut ajid Join metal with a veiling torch. 

Jielj^ people make travel arrangements. 

F a c t c-r V - Pe r s or. a 1 -S e rv J c c 

Help sick people in h rspitals keep busy by wing tb*:m 
Lot to make thir;gs. 

Do things for sick ^.orle in he pito i.c "> j Vlo afjustir.g beds 
to rake them cevLfor table. 

Do I’.air styling.. 

Help t'oer fa'^dlies that have problem:. 




I'actor VI --j.abca atory-Tcchniciaj. 

Sterilize laboratory instruments. 

Vi’ash and set up laboratory glassware useo in srientific 
experiments . 

Head cUuIl am aljust switches in a research laboratcry. 

92 



.81 

.75 

.78 

.68 



.61 



.84 

.75 

.5*3 



.70 

.61 



• y ^ 

. 



.6;> 

.50 

.50 

.85 

.77 






.79 

.?6 



.18 
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Factor II is umn.sts,k8,bly similar to the Outdoor cluster., but also 
picks up an additional Icrwer level occupation (item # 13 ), in support of 
the contention that the percepbion of these^occupations is not on the 
basis of their "outdoor" quality but their menial level. Thus, it is 
designated as Law*- Level ( Outdoor ) Tasks * 

Factor III is identified fairly readily as an Office-Clerical factor. 

Of interest is that the sterilizing of insti-uments is seen in the Office- 
Clerical context rather than in a laboratory-technical setting. Ihe pos- 
sibility IS suggested that the picture and task statement are confusing or 
ambiguous to the male enrollee. The most easily defined factor of the 
group IS Factor IV which is termed a Technical Skills factor and along 

with Factor III constitutes the two clusters closest to the ones derived 
for the COPS measure. 

The major item defining Factor V (i.e., the highest loading) is the 
one that pictures an individual most appropriately considered as a social 
worker., talking to a famiily (item #21). The remaining items fit a pattern 
of job tasks that involve dealing with people directly and helping them or 

selling them something. The dimension seems best considered as a Person al 
Service factor. 

For females, the group of factors is markedly similar to those defined 
for males. Thus, in Table 10 Factor I is seen to be a Clerical cluster. 
Factor II a Lowjlev el Occupational _( Outdoor) cluster and Factor V a Fersonal - 
Service cluster (in direct correspondence to the male sample). The Low- ' 
Level Tec^i c al Skills (Factor III) and the High-Level Technical Skills 
clusters are also counterparts in some respects to the High-Level Technical 
and Busi^ and ^toical-Skills clusters found for mles. The additioml 



93 






factor (Factor VI ) f :>r femles is unique and is designated as a Laboratory 
T eclinician cluster^ s^nce it obviously loads on several tasks that are 
carried out in a laboratory setting. 

p-arther development of items and grouping of tasks into occupational 
clusters for scoring purposes should take into account the groupings sug- 
gested by these factors rather than relying exclusively upon clusters 
derived from the California Occupational Preference Survey^ or any Treasure 
developed with a i.iiddle -class sample. 

Vocational Aspirations and Plans ; From the item characteristics of 
these two measures (regardless of the underlying status continuum) it is 
evident that item means and variances show reasonable patterns of response. 
Hcr.':ever;, use of the measures for analysis purposes was dependent upon sum- 
riiaiy scores of some sort. In this case_, the built-in status continuum 
•with item pairs at eight stat'^s lev^els and the use of identical occupations 
for the VA and VP allowed for the derivation of the three scores 

ut.ili?cd--i . e . , disci^epancy of occupational choice over the status pairs 
(VA and VP Discrepancy scotts) and the difference between degree of occuia- 
t-^onal desii'abj'l ity in terms of aspirations and plans (VA-W Difference 
score). In their re.'* '=».t ion ship to other r.rasu/‘es of tl.c battery and ■*:hc 
criterion ratings., their role was not an outstanaing one. Primarily,, 
there v;as no significant validity achieved for a*-'y of three scores 
(althoijgh the correlations of these scores with the criteria al::w st all 
shvw a tendency toward a negative rePationrhiy)) . A.mo:ig the males the VA-V5' 
Difference score, v:hicPi contributed significantly to interpi'etation of 
cev.jral of the factors, is rela':cd to a rvrrber of r.easiu'cs in the battery 
in a logical way aTs.i would appear to I'.old some iror'jise for use with this 
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group. For females^ on the other hand^ the VA Discrepancy score seemed to 
play a more significa nt^ rol e in terms of its relations "to other measures. 

Reliability of the two measures (split-half comparisons based upon 
matched item-pairs at each status level) was found to be acceptable for a 
scale with so few items and to indicate a fair degree of consistency in the 
perception, by enroUees, of the underlying status continuum*. 

Farther evidence of logical responses to the two measures can be ob- 
tained from more detailed analyses. Examination of the present data can 
be based upon logical hypotheses and on the basis of prior research find- 
ings. Thus responses on the Vocational, Aspirations and Vocational Plans 
measures would be expected to show generally hi^er scores for aspirations 
than for plans (Empey^ I956). Item means should also show a progressive 
increase with an increase in underlying status level--espeeially for aspira- 
tions . 



The item means for Vocational Aspirations and Plans are presented in 
Table 11. 



Table U. 

Vocational .Aspirations and Plans: Mean Scores by Status Levels 






Male 




Female 






Aspirations 


Plans 


Aspirations 


Plans 


Status Level 


Mean 


Mean 


Mean 


Mean 


1 


1.4-7 


1.64 


1.49 


1.53 


2 


2.18 


2.08 


2.09 


1.74 


3 


2.28 


2.09 


2.57 


1-75 


k- . 


2.62 


2.27 


2.5?' 


2.27 


5 


2.94- 


2.66 


2.92 


2.^ 


6 


2.98 


2.4o 


3.31 


2.65 


7 


2.66 


1.98 


3.26 


2.51 


8 


5.22 


2.01 


2.74 


1. 65 
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Aspirations are forxid^ as one niig'^t expect^ to be consistently higher 
than vocational plans for botli mles and females. But the point of interest 
is the progressive increase in mean values for aspirations with an increase 
in status level. Obviously occupational desirability^ in terms of aspira- 
tions^. is tied closely to a status continmim for this group. For vocational 
plans, hovrever, the mles shcA': a distinct do'^mward shift (tovrard greater 
"reality"?) v;ith the highest mean value at status level 5 (plimriber; Auto 
Mchanic) while there is less of a change for females in this respect. For 
the fem;ale sample the highest mean values for plans and aspirations tend to 
occn' at about the same status level--!, e. , level 6 v;hich defined by the 
occupations of "Secretary" and "N-ju’se.” The results might be interpreted 
to mean that females simply express job aspirations that are closer to their 
plans. I'wever, there is a possible artifact in the status scale which pre- 
vents a more orderly progxession of femle aspirations from lo;-; to high 
status choices. The s'oggested difficulty lies in the general lack of occu- 
pations at the highest status levels that tend to be perceived as customar- 
ily occupied by iromen (c.g., judge, la.^yer, engineer) end is equally true 
for the two occupations finally selected for the pic sent measures - -'^Doctor" 
aiii "Scientist." Thus, the unusually sliarp re\t-rsal in status perception 
of both plans and aspiratio.ns for the mean values of status -level 8 might 
result from sucli occupati cu'.s being vievxd as totally outside the realm of 
possibility by femaJ.es. 



V. c'lif I'TSTi'hs i \ i:d kfcom:k::datio:;s 



Any concl\3 sions or broad gcr'jcralizations dram; from rcsi;lts based -upon 
the initial use of a batt^-ry of some 3^ differeiit evaluation measures (and 
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two cx'iterion scales) should be considered qualification and caution* 

Exceptions to generalizations will not be difficult to find for an individ- 
ual measure j or in some limited phase of the data analysis. Nevertheless^ 
keeping in mind the purposes of thi*^ study and^ particularly , its preliminary 
or exploratory nature^ there are a number of reasonable conclusions that 
follow from the results. 

(1) Eirst, and as an overall conclusion, it would appear that the 
battery of measures sho^^s potential value for evaluation purposes (and 
possibly other purposes) with a disadvantaged^ adolescent group that pos- 
sesses relatively low verbal- reading levels. In general, the measures 
would seem to hold somewhat more promise for use with the N.Y.C* out-of- 
school male eni'ollees than v;ith the ferriales. Although^ on the basis of 
the present data, a nunher of the measures can be considered equally useful 
for both groups* In essence, when the content of the measures is con- 
sonant with the liirdtations jn verbal skills and culL.ura.lly levant to th«. 
group being evaluated, the pattern of responses and the resulting statisti- 
cal indices are logical and interpretatle. 

(d) As to .mure specific conclusions; despite the l^omogeneity of the 
c-nrollce saxpd.e (primarily in ter:;\s of cultural backgroinu and race) the 
chaiactevistics of the items use! for these r.cassres werC' found to be 
acceptably adequate in terr.^s of mean scores, variability and itcn-total- 
test correlations. V/hon the test mterial is meaningful, adequate varia- 
tion bctv.'cen individuals is possible from a su-called ^'homcgoieous '' saripJe. 

(i>) Sirriilarly, the reliability (internal con.-islency) estimates lor 
the various measures v.*erc at levels that indicate sufficient consistenev 
of resporise to allow for malti-}'Ur]:'ore utilisation of the battery (i.e. , 
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for predietior. of future perfo-imnce to some extent and for gruidanee and 
placement purpcsejs.. as vrall as for ev‘ 5 a.uaticn). Such consistency lends 
further support to any contention that. the items of these measures d.c com- 
mjfnie.ate their eonterit or meaning to a :reasonable degree- Findings of 
adequate reliehiilty are oj;‘ particular' importance here.; since they su:ggest 
the ‘possibilLlty of maintaining the relatively short test length desirable 
for a disadvantaged, group. 

(4) An attempt to determine validity for each measure, based upon 
criteria consisting of counselor and wrk supervisor ratings results in 
rather low to moderate ^but statistically significant) relationships be- 
tween a number of the measures and these e;^aluations . In addition, the 
patter.n of these relationships is fairly logical- HoweveT*. the criterion 
evaluations themselves are open to question in certain respects since they 
ci*:?e highly gerl■erali:^;ed (i-e-., they indicate a large ^halo*‘ effect) and 
produce ovei'ly l,e:nient judgments., especi:al-ly fo^:- the female enrollees- 
Changes in instructions and administration of the rating scales ai’e felt 
to be *i*equired if they are considered for future use as criteria. Purthe.r- 
mo.re. counselors and wo::-.'k supe:rc>iscrs dc not tend to agree particularly 
well in their evaluations of -what constitutes a ''suitable’'' or ‘ 'unsu.it abl.e‘‘ 
eiurollee. Ob“\/iously, there l.s the unanswered, question of the util.ity of 
counselor and wo;rk super^/'iso.r ratings as c:/:itaria and whether these evalu- 
ations. in turrip bear some reasonabl.e relation to later enrollee -irocational 
success. ‘‘ If it is si^ssumed, for purposes of the present stud.yj that these 
ratings constitute acceptable criteria^^ then the pattern of validity coeffi- 
cients found would indicate that a number of measures in the battery show 
moderate predictive validity. 



(5) The pattern of intercorrelations oeween the irieasm-es of the 
battery are readily interpretable (implying a degree of construct validity) 
and reinforce the general findings of response consistency on the part of 
the enrollees- The factors^ or groupings of intercorrelated measures, that 
help to define the underl3dLng dimensions being measurejd, are found to be 
confined largely to an intellectual skills area of performance (i-e.», 
measures requiring particular knowledge or some degree of verbal reading 
skill) and several attitudinal dimensions- However, there is some degree 
of overlap for these, such that certain of the attitudinal measures and 
intellectual-ability measures are somewhat related- 

When the factors that make up the battery are considered along with 
the criterion ratings, it is clear that the male enrollees are evaluated 
(by counselors and work supervisors) primarily on the basis of intellectual 
capabilities (i«e., measures that require 'reasoning*' or following verbal 
directions) whereas females tend to be evaluated on the basis of an atti- 
tudinal conformity component (primarily in the work situation). Thus, 
certain of the measuz-es might be expected to perform differentially as pre- 
dictors of later success for the male and female enroLlees- 

(6) Of the measures in the battery, the most effective single one — 
for those categorized as intellectual-ability measures was the test 
designated as ”Job Seeking Skills "--which required interpretation of 
employment want-ads and portions of a job application blank- Of the atti- 
tudinal measures the most promising possibilities are shared by the mea- 
sures designated as "Motivation for Vocational Achievement" and TJob 
Holding Skills" which deal Td.th feelings about the importance of working 
and how one should behave on a job. 
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Fui:;are development or utilization of the instruments in this battery 
undoubtedly require additional understanding of their characteristics and 
certain cautions in their applica,tion. Recommended areas of future con- 
cern^ or needed research effort^ involve: 

(1) Firsts the development of normative data on. a larger and more 
diverse sample of disadvantaged adolescents in Neighborhood Youth Corps 
projects* This seems essential <, if for no other purpose than to determine 
an individual's “relative deprivation in comparison with others who have 
also been denied good homes ^ good neighborhoods ^ good diets^ good schools 
and good teachers" (SRSSI^ 196^). Thus^ it shoiild be possible to compare 
performance for an e.nrollee on any given measure in relation to scores 
made by some pertinent reference groups of disadvantaged adolescents 
(based upon sex^ age^ geographic location^ or other desired subgroupings). 

As part of data collection for normative uses^ the wider application of 
the measures and the opport-anity for reanalyses of resulting data- serves^ 
of course^ to add to their status and utility as formal assessment instru- 
ments • 

Clearly^ appropriate normative data should be obtained from individuals 
with a reasonable comprehension of English^ since it is not feasible to use 
the present measures with those who have only minimal capability with the 
spoken as well as TOitten language* (Translation to Spanish for use with 
Puerto Rican or other Spanish-speaking enrollees is not precluded for later 
development. ) It is also questionable as to whether the present battery 
would be applicable to such unique cultural subgroups as American Indians. 

(2) As a second area of future concern in the use of the measures^, 
there remains a particular problem involving group administration for those 
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t iu'Oj-j-6c 5 aij 3.n Gxc op"!/ ions. j-3.y lcr,r-ve>'bal reading (e.g.^. fourth grade 

or lower)' Separate administration in smll groups may be required for 
tViose enrollees who seem to profit from the oral pres*;:; at at ion; whereas, 
for those at 2 ‘elatively high verbal reading levels (e.g., seventh grade 
level or higher) the examiner's oral presentation can be annoying and they 
tend to move ahead of the group in their responses to the items. Some 
screening, based upon a set of trial items or background information 
(where available) should be considered for this purpose. Conceivably, 
tape recorded instructions --concrollable by the enrollee --could be util* 
ized to deal with the problem. 

( 5 ) Development of alternate forms of the measures coiistitutes a 
third area for future consideration. This is of importance if the mea- 
STjjes are to serve as evaluative instruments for detecting change in 
enrollee characteristics over a relatively short period enrollment in 
IhY.C. Thus, the alternate forms should serve to overcome any effects 
of recall of the material in an evaluation study and also provide a means 
for obtaining test-retest reliability estimates over relatively short time 
periods ( e- • g . , vS c ve r a 1 d ays ) . 

(^0 Ai:other irajor point to be considered in future- application of 
the e va luat i on m.c a s u r ■: s is the d e ve .1 opme nt 0 f r e 1 ev ant, p:o st-IUY.C. cri- 
terion measures of enro31ee vocational ad^iustment. Suitable criteria are 
necessary to provide evidence regarding the characteristics that are beirjg 
assessed by any battery of measures. Validation of the measures against 
longer -range criteria than ratings by counselors or v.urX supoi^vi sors would 
al^o provide needed (-vidence of their accuracy and suitabiJity for a var- 
iety of potential uses by if 7.0. 
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VJider application of the jaeasures in the context, of a youth-work 
training prograifi are recoiriwnck.d. generally^ as the only means to deter- 
mine v.hich laeasm-es ai'C rnOsSt effective and for vrhich situations. There- 
fore the process of developing the measures should be a continuing one^ 
during which their utilisation as evaluative or g^aidance instrwnents adds 
to the knowledge of their applicability and also provides some unifoiTiity; 
cr consistencyj, of research results. Continued use of "ad lib^" infoi'mal 
questionnaires j, as so-called "measures" with psychometric propei’ties that 
are largely unkno'.ni and content that meets the changing needs as perceived 
by each individual investigator, can only provide self-defeating incon- 
sistencies in conducting a progra'.i of evaluation and research. 
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Ggneral Directions 

[To follow informal introductory remarks by Examine^ ’’Vie appreciate 
the fact that you’re willing to help us by trying out some new kinds of 
questions. These are in small booklets like this one. ^ sample*^ 

In most of the booklets there are also pictures along with the questions. 
Everything should be pretty easy to understand - but I'll read almost all 
the things ’written in the booklets to you, so you don't have to worry too 
much about kno’wing all the words. A lot of the questions have to do with 
how you feel about different jobs; what you think people are like; how you 
feel about yourself and how you feel about the Neighborhood Youth Corps. 

Let me know if there is anything in them that's putting you down or would 
be insulting to answer ^ because you don't have to ans;;er anything that you 
feel we have no right to ask. But we think the questions are pretty straight 
and we hope you'll have some fun working out a lot of them. 

The answers that you give to any of the questions will never be seen 
by anybody at an M.Y.C. project and none of this is going uc be used to 
affect anything you do in N.Y.C. So you're free tc ans*wer the way you 
really feel - you don't have to ir^press anybody.^' 




12G 



JOB KNOWLEDGE (JK) 



I. ADfflNISTRATIOM 

Exaniner Reads : 

, Instru "itions 
. Job Titles 
, Item Stems 

, Item choices (Tools pictured are not to be nam.ed by the examiner) 



II. ^ STRUCTIONS 

”Miat we would like to find out with the questions in this booklet is how 
much you know about different jobs. You’ll see pictures of all sorts of jobs 
and the name of each job like a ’Dock V/orker’(or Longshoreman).” 
jHave enrollees turn to first item in bookletTj 

’’Right after the picture of the Dock b'orker ai^e three questions. The first 
one says: ’How much schooling do you think you’d need to get this job?’ If it’s 
a college degree mark the first box. If you need a trade school certificate 
mark the second box.” ^£ontinue reading item stems, item choices and job 
titles for all itemsTl 
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VOCATIONAL PIANS AND A SPIRA TIONS (VP, VA) 



I . AP mNISTRATIQN 

ExairAn er Reads ; 

, Instructions 

• Job Titles 

• Job Descriptions 

• Response Alternatives 



II. INST RUCTIONS 

(a) Aspirations : 

”In this booklet you^ll see pictures of different kinds of jobs that 
people have. With each picture it tells the name of the job; then it says a 
little about kind of work a person does on that job." 

"Let's suppose you could have any ^iob you see here, if you wanted it. 
How would you feel about v^orking at each of these jobs. Like an auto mechanic. 
You tell how you feel about that job by raking a nark in one of the four boxes 
ne.d: to the picture. If you feel its a great job; the kind of thing you would 
like — draw^ a line through the first box. If its pretty good and you wouldn't 
mind doing this — mark the second box. If it*s not the greatest, but you would 
do it ^f there was nothing better — mark the third box. And if its a i-otten 
kind of job that you vroul'^n't like at all — mark the last box." |^aaminer: 
Points out each response stal:cm',^nt in bookletT| 

O 

ERIC 
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VOCATIONAL ?1MS AND ASPIRATIONS (C^ontinued) 

’’Now look at this first picture again. It shows a crane operator (Doctor 
for female booklet) arid it sa^^s ’Operates a crane to lift heavy things,’” 
’’Okay, mark the box that shows how you feel about that job.” 



pontinue with remaining items ^ reading response alternatives for first 
several items or until it is apparent that the exaininees fully understand 
the response continuum. Read all job titD.es and Tob descriptionsTj 



(b) Plans ; 

”Now we’re going to look at the same pictures again, but this time I want 
you to do something a little different. Think about wiiat you might be able 
to do after you leave the N.Y.C, program and what kind of jobs you would look 
for over the next few years. Then look at the first picture that shows the 
crane operator. If you think the odds are that you would do something like that — 
then mark the first box. If you think there's a fair chance you would wind up 
in this job — mark the next box. If you think this job is possible, but you 
don’t think chances are too good that you would really do this — then mark the 
third box. If this is way out for you and you wouldn’t even look for this kind 
of job — put a mark in the last box down.” 

"Now look at this first picture again. It shov/s the crane operator and 
it says, ’Operate a crane to lift heavy things.’” 

”Okay, mark the box that shows how you feel about that job,” 

(Continue with remaining iterjis; reading response alternatives for first 
several items, or until it is apparent the examinees fully understand these. 

Read all job titles ar.d Job descriptionsTj 





ATTITUDE TOWARD AIFTHORITY (AA) 



I- AD MINISTRATION 

Exaniiner Reads 

• Instructions 

• Verbal material accompanying each pictorial scene 
. Item Stems 

• Response Alternatives 



(Only a simple descriptive statement by the examiner is acceptable in referring 
to any pictorial content - e.g,, ’’the cop talking to the g’o^y^ sitting on the 
grass. ") 



II. INSTRUCTI ONS 

"What we would like to find out now is how much you know about the way 
people would act with each other in different situations, Youni see pictures 
of people in a lot of eveiyday scenes - like the first picture in this booklet 
that shows a guy talking to a teacher who says ^You'll have to do better^ your 
marks aren^t so good,^ He answers - (Right below the picture) - ’1^11 try to 
study more and get someone to help me with the work.^" 

"New over on the right it says, 'Statistics would show that the number of 
guys who give this answer to the teaclier is: Z out of 10 guys, 3 out of 10 guys, 
out of 10, 5 o\:X of 10 or 6 out of 10 guys.'" 

"Mark the box that tells best how many guys you think would give this 
answer to a teacher*" 

O 




j^ead 



remaining itemis in the sam.e way 
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SELF- (SE) 



I, alm:i mstration 

Exaipiner Reads 
. Instructions 

. Verbal teterial in Pictorial Scene 
. Response Alternatives 



II. INSTRU CTIONS 

you T 11 see here are pictures about everyday things that happen to 
young people. Maybe even some that cculi happen to you. Like the picture 
i^ere you see a guy (girl) who’s applying for a job and the man at the desk 
says ’So you think you want a job here!' Make believe that the young person 
in the pictiu'e is you . Below the picture are some of the ways you might feel 
if it really were you standing there.'* 

”Put a line in the box next to the way that tells best how you would feel. 
If you would think ’They would probably tell me I'm someone they can’t use' - 
put a line in che first box. If you would th:’nk 'They would offer me a pretty 
good job' - mark the second box (where it says that). If you would think 'They 
niight offer me some kind of job - but nothing too good' mark the thi_rd box.” 
"New let’s look at the next picture." 

^ntinue reading response al.ternatives in the sair.e manner as above. The 
brief description of the pictorial scene is only to be used for pui*poses of 
referring to the item, e.g., "The girl standing at tne counter" Do not 



'elaborate on the pictorial content unnecessarily 
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ItJTEREST IN VOCATIONAL TASKS (vi) 



I. ADMINISTRATION 

Examiner Reads : 

. Instructions 
. Job Tasks 

. Response Alternatives 
II. It TmUCTIO NS 

’’This next set of pictures shows different things people do on jobs. Kith 
each picture, it tells what t.he person is doing. Some of these things you 
might like to do. Some of them you might not like at all. For some ;)^ou might 
not care too much one way or the other," 

"For example, look at the first picture where it says *Be responsible for 
taking care of office files, ^ and that's what it shows the guy doing." 

"Now think of how you would feel about doing that yourself. Don't worry 
about whether you kjiow how to do this or if you have enough school or training — 
just hox\- you feel about doing it. If this is the W'orst and you couldn't take 
doing it at all; jus: mark through the first box (where it says that). If its 
not just what you would like but not the worst - put a rrark through the next 
box, If its pretty good and you wouldn't mind doing this - mark the third box 
down, And if it's great - just what you w*ould Tike to do, put a mark through 
the last box." 






INTEHEST IN VOCATIONAL TASKS (Continued) 



"Okay, now look at the next pict’ure, it says. . 

I^ntinue as for the first iten. Read tasks: read response al- 
ternatives only until examinees readily grasp the fo’ur point response scal^ 
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DEFERRED GRATIFICATION! (DG) 



I. ADmiSTKATIQN 

E:caini net* Reads 
. Instructions 
, Item Stems 

*< 

. Response Alternatives 



II. IKSTRUCTIOMS 

’^The things witten in this booklet tell some of the ways people feel about 
getting along in life. We would like to see how good you are at guessing what 
most people might think about these. 1*11 read each one vrith you and you’ll 
tell hov you think people would feel about it. Like the first one on the page 
that says . . . *It pays to listen to advice your parents give you.”' 

’*If you think almost nobody would feel this way mark the firs t box. If you 
think only a few people nught agree with this then mark the second box. If you 
think some people would agree mark the third box. If a lot of people would 
feel this way mark the fourth box and if you think most everybody would agree 
mark the last box. (Mark only one box - the one that’s closest to the way you 
think it is.)" 

"Now the next one says ..." 

^ontinue for remaining 19 statements. Once the response continuum is 
apparent to the enrollees it is not necessary to repeat the reading of all 
O esponse alternatives for every iteirnl 

ERIC 
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MO TIVATION FOR VOCATIONAL ACHIEVEMEN^f (mk) 



1 . .AI)MIN 1STH\TI0N 

FiXairdner Reads 
. Instructions 
, Item Steins 
, Response Alternatives 



II. INSTRUCTIONS 

’’The things ve're asking about here have to do with what you think about 
jobs and about going to work. So we wrote dc*-ni what people sometimes say about 
jobs and about working and we want to know how you feel about these, 

''The first one says *I would go to school at night for special courses if 
it meant I could get a better job.’ Under that it says ’Nearly Always;’ ’Most 
of the time;’ ’Sometimes this is so;' 'Very few tirries; ’ and ’This is almost 
never so, ’ 

"Put a line through the box next to the way that tells best how you feel." 
In struct examinee to respond to the first item. Continue with oral presentation 
of next 11 items^ 

"Now these next ones are a little different. Here in the booklet it says 
'Let's say you have a job that you know was going to be a much better one if 
you stuck it out, ’•'ow far would you go to stay with it?'" 

"Would you stay with it if..." 

item ^ ems and responses for each of the remaining seven item^ 
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JOB SEEKING AND JO B HOLDING SKILLS (0S;Jll) 



I. ADMNISTRATION 



Examiner Reads : 

. Instructions 

. Job- seeking- skills item stems and responses (Do not read material 
in want ads or any content of application blanks) 

. Job holding skills item stems; item responses and verbal material 
in pictorial scenes, ( Simple description of pictorial scene is 
permitted - e.g. , ''boss talking to a guy coming in to work")- 



II. INSTRUCTIONS 

"V^e're interested in how you feel people should act on a job - and what you 
might do on a job when certain things happen. For exair^ple, it says here [Read 
first ite^ , If you think you give them one week^s pay - mark the first box. 

If you give them - whatever you can afford - mark the second box remaining 

alternatives: Read next verbal item“.| 

^'Now we^re going to look at some job 'want ads' - like the kind that you 
see in the newspaper when someone wants to hire people for Jobs. Read each want 
ad on your own jXllow approximately 3^ seconds for reading of aTj - then I'll 
read the questions to you that ask about \diat's in the want ad." ^Read the nine 
want ad item^ 

"These next questions are about parts of an application blank - like the 
kind you fill out viien ycu go to get bird for a job. Look at this part of the 
application blank and the things it asks you to fill out." |^low approximately 
O ^5 seconds to look over blank. Then read each item stem and alternatives.) 

ElUC 136 



^JOi: ;'V7 lKINVt nou^ino spills (ContlnucO^^ 



■^IJow ,’xok aC the next application blank that some g^iy (giri) filled in. 
Reaa ti nt 073 r.'^ j^llow about one minute to read material. Read each item 
ster. and alternatives^ j^ntinue on to Job Holding items - read all verbal 
imate<'ial '.ncluding vez'bal portions of pictorial scenesTj 
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PRACTIC AL REASONING (PR - Z,M,F) 



I. ADMINISTPJiTIO N 

Examiner Reads 

♦ Instructions as ^iven in each Booklet 
. Item Steins 
. Iten Responses 



II. INSTRUCTIONS 

•'Now "W-’re going to go through questions in three booklets that ask you to 
do some of the things you might have to do on certain jobs. In one you'll be 
asked to follow divectici.s for delivering things to people. In another^ you'll 
be asked about sorting mail in a post office using ^ip codes and in a third, 
you'll answer questions about filing cards." 

"let's look at this first booklet. Open it up and we'll read the directions." 
^dminister Map Reading subtest. Examiner reads directions and item^ 

"Now open this next booklet." 
j^dmnister Zip Code subtesF^ 

"Now open the third booklet. Bull off the envelope on the back of the booklet 
and take out the cards inside." 

^dninister Card Filing subtes^ 



Do not read complete details of map, or zin-code chart. 
O ily for Practical Reasoning file cards is permissible. 

ERIC 



The reading of headings 
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ENROLLEE RA^ ING SCALE (ERS) 



ADMirJISTRATION 
Examiner Reads 

* Instructions 

* Item Stems 

, Response Alternatives 

II* instruc tions 

'^On the pages of this booklet are some of the things that enrollees say- 
are important about N»Y.C* We would like to know how you think these comments 
fit the N.Y.G. program* For each coment you tell how much you think it fits 
your N*Y.C. project by putting a mirk in the box next to the way that tel?-S 
best how you really feel. Like the one on the fiist page that says ..." 

'^If yo - think this is alrays the '.vay it is in N.Y.C. mark the first box. 
If this is pretty much the way it is mark the second box. If it^s sometires 
so, mark the third box. And if you feel this is never the way it is in your 
N.Y.C. project, mark the last box. 



O 
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N.Y.C* Project 



COUNSELOR'S RATING SCALE 
Neighborhood Youth Corps 

Counselor’s Naine Name of Enrolloe 

Being Rpted 



On this sheet are 11 statements about enrollee behavior that counselors consider 
iirportant in determining how well an enrollee is coming along. We would appreciate 
your evaluation of hcv these apply to this enrollee. 



Please read each statement carefully . Then circle one of the numbers 1 to 5 that 
best indicates how the statement applies to the enrollee. 



1 



2 3 



I 5 



This describes 
the enrollee 
perfectly 



This is 
true most 
of the 
tLme 



Sometimes 
this is 
true of the 
enrollee 



This is The enrolloe 

not is not like 

usually this at all 

so 



Circle one 



!• Pays attention to good grooming and dresses appropriately. 1 ? 3 Li 

Is n^t very open about discussing personal end job i 2 } h 

problems. 



3. Shows a lot of resantment and hostility . 



U. Is cooperative and willing to listen to advice. 

5. Makes realistic plana about future jobs. 

6. Shows little poise or self-assurance. 

7. la coherent in expressing himself (herself). 



1 2 3 U 

123a 

1 2 3 ii 

123a 



3. Is motivat'd to want to work and expend effort. 1 2 3 u 

9. Does not show good day-to-day planning so thau ne (she) 1 2 3 ii 

can handle the job (let’s home life interferOj for 
example). 



10. Indicates a wllllngnaas to enroll in school or some sort 1 2 } h 

of training on a part-time basis. 




Shuv;s qualitios that indicate he (she) will do woll 
after leaving N.Y.C* 



1 2 3 Ii 



5 

r' 

? 

5 

5 

5 

5 

5 

5 

5 

5 

5 
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NcY*C. Project 



WORK SUPERVISOR'S RATITO SC/ilE 
Neigbbor)>ood Youth Corps 

Work Supervisor's Name Nwne of Enrollee 

You Are Rating 



Below are 10 statements about thr.ngs that work supervisors consider Irportant 
when it comes to how the enrollee is doing, Wc* would appreciate your telling 
us how each one applies to this enrollee* 

Please read each statement carefully . Then put a check in one or the five 
boxes to shew how that statement fits the enrollee that you're rating. 



1. KNOWS HOW TO POJXCW INSTRUCTIONS PROPERLY, 



This describes 


This is 


true 


S- riet5Tr.es this 


This is not 


The enrollee 


just how the 


most of 


the 


is true of the 


U3v-'*lly so 


is riot like 


enrolles is 

n 


time 

□ 




enrollee 

□ 


□ 


this at all 

□ 



2. RESENTS TAKING ORDERS FROM THOSE WHO SIMPER VISE HIM (HER), 



This describes 
just how the 
enrollee is 

□ 


'Phis is true 
most of the 
time 

□ 


Sometimes this 
is true of the 
enrcllee 

□ 


This is not 
usually so 

□ 


The enrollee 
is not like 
this at all 

□ 


3. KNOWS HOW TO DRESS 


RIGHT FOR THE 


JOB. 






Tliis describes 


This is true 


Sometimes this 


This is not 


The enrollee 


j\jst how the 


most of the 


is true of the 


usually so 


is not like 


enrollee is 


time 


enrol] ee 




this at all 


□ 


□ 


□ 


□ 


□ 



li. TAKES SOME PRIDE 


IN THE WORK AMD 


DOESN^T JUST RUSH 


THROUGH TO GET IT 


FINISHEDc 


This describes 


This is true 


SometimLes this 


This Is not 


The enrollee 


Just how the 


most of the 


is true of the 


usually so 


is not like 


enrollee is 


time 


enrollee 


this at all 


□ 


□ 


□ 


□ 


□ 










CAN'T KEEP BJSY 



5. HAS TO BE TOLD WHAT TO DO EVERY MINUTE OR HE (SHE) 



This describes 
just ‘how the 
enrollee is 

□ 


This is true 
most of the 
time 

□ 


Sometimes this 
is true of the 
enrollee 

□ 


This is not 
usually so 

□ 


Tl'ie enrollee 
is not like 
this at all 

□ 


6. GETS ALONG WITH OTHERS ON THE JOB. 








This describes 
just how the 
enrolleo is 

□ 


This is true 
most of the 
time 

□ 


Sometimes this 
*is true of the 
enrollee 

□ 


This is not 
usually so 

□ 


The enrollee 
is not like 
this at all 

□ 


7* CAN’T GET TO WORK ON TIKE* 








This descri.baB 
just ho^ the 
enrollee is 

□ 


This is true 
most of the 
time 

□ 


Sometimes this 
is true of the 
enrollee 

□ 


This is not 
usually so 

□ 


The enrollee 
is not like 
this at all 

n 


J. SHOWS SOKE INITIATIVE IN TAKING ON A PIECE OF VK)RK. 






This describes 
just how the 
enrollee is 

□ 


Ihis Is Vme 
most of the 
time 

□ 


Sometimes this 
is true of the 
enrollee 

□ 


This is not 
usually so 

□ 


The enrollee 
is not like 
this at all 

□ 



9. DOESN'T MAKE TROUBLE ON THE JOB. 



This describes 
just hew the 
enrollee is 

□ 


This is t*oie 
.most of the 
time 

□ 


Some times this 
is true of the 
enrollee 

n 


Tins is 
usually 

□ 


not 

so 


The enrollee 
is not like 
this at all 

□ 


10. ASKS QUESTIONS 


IF PR0BL£K5 COME UP 


-- DOESN'T JUST GO 


AHEAD km 


DO 


T^IE JOB WONG- 


This describes 


This is true 


Sometimes this 


This is 


not 


The enrollee 


just hew the 
enrollee is 


most of the 
time 


is true of the 
enrollee 


usually 


60 


is not like 
this at all 


□ 


□ 


□ 


□ 




□ 



11. SHOWS THE KIND OF QU-ALIHES ON THE JOB THAT WILL HAKE HIM A GOOD WORKER AFTER HE 
LEAVES N.Y.C. 

This describes This is true Sorietimes this This is not The enrol lee 

just how the most of the is true of the usually so is not like 

enrollee is time enrollee this at ail 

□ n □ □ □ 
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